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ABSTRACT 



Three discrete reliability growth models using fractional failure reduction, referred 
to as failure discounting, were developed to estimate changing system reliability. Each 
of the models is designed for use when testing is performed until a fixed number of 
failures have been observed and attribute data, success or failure, is available for each 
trial. The first reliability growth model applies failure discounting to the maximum 
likelihood estimate for a proportion. The second and third models use a modification of 
an exponential reliability estimate employing linear regression and a weighted average 
technique respectively along with failure discounting to track changing reliability. Two 
failure discounting methods were used with each reliability growth model. The first 
method reduces past failures by a fixed fraction at a fixed interval. The second method 
uses the upper confidence bound for the reoccurrence of each failure cause as the 
discounted failure value. The performance of the reliability growth models with 
varying reliability growth patterns was evaluated with a Monte-Carlo simulation. 



THESIS DISCLAIMER 



The reader is cautioned that computer programs developed in this research may 
not have been exercised for ail cases of interest. While every effort has been made, 
within the time available, to ensure that the programs are free of computational and 
logic errors, they cannot be considered validated. Any application of these programs 
without additional verification is at the risk of the user. 



5 



TABLE OF CONTENTS 



I. INTRODUCTION 10 

II. DISCRETE RELIABILITY GROWTH MODELS 12 

A. BACKGROUND 12 

B. DISCOUNTING METHODOLOGY 13 

1. Straight Percent Discounting Method 14 

2. Lloyd Failure Discounting Method 17 

C. RELIABILITY GROWTH MODELS DESCRIPTION 21 

1. Maximum Likelihood Estimate With Failure 

Discountmg 22 

2. Exponential Single Phase Reliability Estimate 27 

3. Exponential Regression Discrete Reliability Growth 

Model 29 

4. Weighted Average Discrete Reliability Growth Model 32 

III. DISCRETE RELIABILITY GROWTH SIMULATION 36 

A. MODELING ASSUMPTIONS 37 

1. General Assumptions 37 

2. Reliability Growth Pattern Assumptions 37 

3. Failure Cause Determination Assumptions 39 

3. SIMULATION ALGORITHM DEVELOPMENT 40 

1. Simulation of a Reliability Growth Pattern 40 

2. Generating Test Data From Known Reliability 41 

3. Application of the Discounting Methods and Reliability 

Growth Models . .' 43 

4. Statistical Summary of Model Results 45 

C. CAPABILITIES OF THE MODEL 46 

IV. EVALUATION OF RELIABILITY GROWTH MODELS 47 

A. GENERAL RESULTS 48 



6 



B. WEIGHTED AVERAGE RELIABILITY GROWTH 

MODEL 48 

C. MAXIMUM LIKELIHOOD ESTIMATE WITH 

FAILURE DISCOUNTING 48 

D. EXPONENTIAL REGRESSION MODEL 52 

V. SUMMARY AND CONCLUSIONS 63 

A. SUMMARY 63 

B. CONCLUSIONS 64 

C. RECOMEND ATIONS 65 

APPENDIX A: DISCRETE RELIABILITY GROWTH MODEL 

SIMULATION 67 

1. FORTRAN CODE FOR THE SIMULATION 67 

2. SAMPLE SIMULATION INPUT FILE 78 

3. SAMPLE SIMULATION OUTPUT FILE 79 

APPENDIX B: GRAPHICAL RESULTS OF THE RELIABILITY 

GROWTH MODEL SIMULATION 85 

1. MEAN ESTIMATES FOR EACH ESTIMATOR WITH 

CONSTANT PARAMETERS 85 

2. ESTIMATOR SENSITIVITY TO FAILURE 

DISCOUNTING PARAMETERS 107 

3. ESTIMATOR VARIABILITY ( MULTIPLE BOX PLOTS ) 116 

LIST OF REFERENCES 126 

INITIAL DISTRIBUTION LIST 127 



7 



LIST OF TABLES 



1. SAMPLE TEST RESULTS 16 

2. STRAIGHT PERCENT DISCOUNTING OF EXAMPLE DATA ( 

TABLE l ) 18 

3. LLOYD DISCOUNTING OF EXAMPLE DATA ( TABLE 1 ) 21 

4. MLE WITH DISCOUNTING OF SAMPLE TEST DATA (TABLE 2) 25 

5. MLE EXPECTED VALUE 26 

6. ESTIMATE EXPECTED VALUE FOR MLE & EXPONENTIAL 

ESTIMATE 28 

7. EXPONENTIAL REGRESSION ESTIMATE COMPUTATIONS 32 



8 



LIST OF FIGURES 



3.1 Regression Estimate of Y 44 

4.1 Reliability Growth Models Low Constant Reliability No Discounting 

Applied 49 

4.2 Reliability Growth Models Low Constant Reliability Std. 

Discounting Applied 50 

4.3 Reliability Growth Models High Constant Reliability Std. 

Discounting Applied 51 

4.4 Reliability Growth Models Low Reliability Moderate Growth Std. 

Discounting Applied 52 

4.5 Reliability Growth Models High Reliability Moderate Growth Std. 

Discounting Applied 53 

4.6 Reliability Growth Models Low Reliability Rapid Growth Std. 

Discounting Applied 54 

4.7 MLE With Discounting Lloyd Discounting Method 55 

4.8 MLE With Discounting Straight Percent Discounting Method '..... 56 

4.9 MLE With Discounting Estimate Variability at Low Constant 

Reliability 57 

4.10 MLE With Discounting Estimate Variability With Increasing 

Reliability 58 

4.11 Exponential Regression Model Lloyd Discounting Method 59 

4.12 Exponential Regression Model Straight Percent Discounting Method 60 

4.13 Exponential Regression Model Estimate Variability at Low Constant 

Reliability 61 

4.14 Exponential Regression Model Estimate Variability With Increasing 

Reliability 62 



9 



I. INTRODUCTION 



Accurately determining the reliability of a newly designed piece of hardware is an 
important part of material acquisition in both the military and civilian sectors. 
Significant amounts of both time and money are spent testing new systems to 
determine if they meet specified reliability requirements. Since most organizations are 
under constant pressure to conserve both time and funds, it is important to develop 
methods of estimating system reliability at minimum cost. When conventional 
estimators for system reliability are used, verifying high reliability requires a great deal 
of test data costing both time and money. This problem is compounded by 
technological advances which produce more complex and highly reliable hardware at 
an ever increasing cost. Developing new and more accurate methods of estimating 
reliability that reduce required testing could result in significant savings of resources. 

Early versions of newly designed equipment often exhibit low reliability. As 
engineering designs are tested and improved the system reliability usually grows 
rapidly. This process continues throughout development phases and into field use. By 
the time a product design reaches its final configuration, a large amount of test data is 
usually available from tests performed on previous configurations, subcomponents and 
parts. Since conventional estimators assume a constant reliability, it is necessary to 
wait until the finished piece of hardware is available to begin reliability verification. 
This practice is not an efficient use of constrained resources because results from 
previous testing are discarded. If all the data available could be used to estimate final 
configuration reliability, the lead time and cost of the finished product could be 
reduced. Reliability growth models serve this purpose. They use data from testing at 
all phases of development to establish a reliability growth pattern. This growth pattern 
can then be used to provide current estimates of system reliability with less testing than 
standard testing programs. 

Reliability growth models can be classified into two categories, discrete and 
continuous models. This paper will address discrete reliability growth models that 
employ failure discounting. These growth models are used to determine reliability for 
systems where test results can only be stated in terms of success or failure, ie; attribute 
data. Failure discounting allows the user to fractionally discount past failures when 
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resultant corrections to equipment are verified through subsequent testing, as actual 
corrections to the cause of failure. The use of failure discounting makes it possible for 
previously linear estimators to track a non-linear growth pattern. 

The objective of this paper is to develop and evaluate the performance of discrete 
reliability growth models using failure discounting. Three different reliability growth 
models are proposed and evaluated with two different methods for discounting past 
failures. Monte-Carlo simulation is used to evaluate each of the three growth models 
under both discounting methods. The simulation uses actual reliabilities to generate a 
random growth pattern. From this growth pattern, test data is then generated for the 
known system reliability at each testing phase. Each of the reliability growth models 
can then be applied to the test data producing an estimated growth pattern. The 
performance of each reliability growth model is evaluated by comparing the estimated 
growth pattern with the actual reliability in each phase. 

The following chapter will address each of the three reliability growth models 
used and the two methods of failure discounting employed. Chapter III contains a 
discussion of the computer simulation used to evaluate each of the models. Chapter IV 
presents the results of the evaluation and Chapter V summarizes the study stating 
conclusions and recomendations for future study. 
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II. DISCRETE RELIABILITY GROWTH MODELS 



A. BACKGROUND 

Three different reliability growth models and two different techniques for 
fractionally discounting past failures are discussed in this chapter. The reliability 
growth models can be divided into two categories. Models using a modification of the 
maximum likelihood estimate, MLE, for a proportion, referred to as the MLE With 
Discounting Model, a model referred to as the Weighted Average Modei and an 
Exponential Regression Model. All of the models may be applied with or without 
failure discounting and are independent of the discounting method. 

Two different methods for fractionally discounting past failures are also 
discussed. The first method is referred to as the Standard or Straight Percent 
Discounting Method. The second was proposed by David K. Lloyd and will be 
referred to as the Lloyd Method. Before the specific details of each growth model and 
discounting method are discussed, several terms need clarification. 

Reliability, R, as referred to in this study is defined as the probability of a system 
successfully completing a single trial or operation cycle, Equation 2.1 . R will be used 
to denote an estimate of the actual reliability R. 

R = Pr ( SINGLE TRIAL SUCCESS ) (eqn 2.1) 

Discrete reliability growth models are normally used for trials which can only be 
evaluated in terms of a binary variable, success or failure, i.e. attribute data. Exactly 
what constitutes a trial is normally defined by the test agency but could include, for 
example, a single rocket engine ignition attempt or a single test firing of a missile up to 
and including target .destruction. To use the reliability growth models discussed here, 
trials must be defined such that they are discrete and can be evaluated as a success or a 
failure. 

A testing phase is defined as a collection of trials, one or more, for which the 
configuration of the system being tested is unchanged. This implies that the actual 
system reliability remains constant throughout a single phase. If a test-fix-test 
methodology is employed, design changes or "fixes" are implemented after each failure. 
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In this case, a phase will consist of one failed trial and all the successful trials leading 
up to the single failure. If on the other hand a test-fmd-test methodology is used, a 
single phase may include numerous failures. Under both methodologies, a phase is the 
collection of all trials between configuration changes. [Ref. 1] 

Because the underlying reliability of the system may be different from phase to 
phase, a new estimate of reliability must be computed for each and every phase. As 
such, R k , denoting system reliability and i? k denoting the estimate of system reliability 
in phase k, will be used throughout the paper. 

Annotating the cause of each failure is critical to all discounting methods 
considered in this study. How the failure cause is defined, is dependent upon the 
judgement of the model's user and the type of system to which the model is applied. If 
the model is used to determine the reliability of a single component, failure cause refers 
to the reason for failure such as a broken cam-shaft or fuel stoppage. If the model is 
applied to determine the reliability of a system of components, the failure cause may 
refer to the failure of a sub-component or the specific reason for failure. To apply the 
discounting methods discussed in the paper, the model user must determine if two 
observed failures are due to the same cause. It will not be necessary, however, to 
enumerate all possible failure causes. The level of detail at which failure causes are 
defined is up to the user and should be determined before any testing is begun. The 
discounting methods studied will be discussed in the next section. 

B. DISCOUNTING METHODOLOGY 

Weaknesses in a system design become known as they cause failures during 
testing. As weaknesses are identified and corrective "fixes" are implemented the 
probability of a repeat failure due to the corrected cause should be reduced and system 
reliability improved. This concept forms the basis for failure discounting. Once a 
system weakness has been corrected and the improvement validated through further 
testing, fractionally discounting the past failure is a method of reflecting the improved 
system reliability in the previously collected data. When using standard reliability 
estimators such as the maximum likelihood estimate, Equation 2.2 

Number of Successful Trials 

R = (eqn 2.2) 

Total Number of Trials 
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only the data from the phase of interest can be used to determine system reliability 
because system reliability is different from phase to phase. If the proper discounting 
methodology can be developed, failure data from all past phases could be discounted 
and made compatible with the data from the current phase. Increasing the amount of 
data available for computing the estimate of current system reliability will increase the 
accuracy of the estimate and reduce the amount of testing necessary to verify system 
reliability. 

Of equal importance to a method for discounting past failures is a penalty system 
that will restore all or some portion of a previously discounted failure if its failure 
cause should reoccur. If a previously implemented "fix" proves not to have corrected a 
design weakness, the discounts applied to its failures will diminished - erased reversing- 
the effect of discounting. A properly designed penalty factor will make the discounting 
method self-correcting. 

In order for a discounting method to be useful, it must be flexible enough to 
handle many varied reliability growth patterns. To achieve this flexibility, discounting 
methods with one or more input parameters are proposed. Regardless which 
discounting method is used, experience and good engineering judgement will be 
necessary to choose input parameters that are compatible with- both the reliability 
estimator used and the proposed test plan, if improper parameters are applied, the 
model may consistently over or underestimate system reliability. The following 
sections describe the specific steps for applying two failure discounting methods 
including examples of their use. 

1. Straight Percent Discounting Method 

The Straight Percent Discounting Method removes a fixed fraction of a failure 
each time a predetermined number of trials are observed without reoccurrence of the 
failure cause. Before a failure is discounted, the design "fix" correcting the failure cause 
should be verified by further testing. This concept forms the basis for the discounting 
method. The fraction of a failure removed, F, is referred to as the discount fraction. 
After the initial occurrence of a particular failure cause, any trial in which the failure 
cause is not repeated is considered a success. The number of successful trials between 
application of the discount fraction is referred to as the discount interval, N. Both N 
and F are input parameters specified by the user. They remain constant throughout 
failure discounting computations. To use the Straight Percent Discounting Method, it 
is necessary to record the number of successful trials since the last occurrence of each 
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observed failure cause, referred to as M. Failure discounting is performed at the end of 
each testing phase using the values for N, F and M. 

The mathematical representation of the straight percent discounting method is 
shown in Equation 2.3 

ADJUSTED FAILURE = (1 - F) INT ( M / N) (eqn 2.3) 

F = Fraction of a Failure to be Removed 

M = Number of Successful Trials Since Last Failure for the Failure Cause 
N = The Discount Interval Expressed in the Number of Successful Trials 

INT is a function which returns the truncated integer portion of its argument. M is set 
to zero at each occurrence or reoccurrence of a particular failure cause. This causes the 
value for adjusted failure to be set at 1.0 erasing any previously applied discounting. 
This also functions as a penalty factor allowing the model to correct itself should past 
discounting prove unwarranted. By altering the choice of N and F the amount of 
discounting applied can be varied from none to total loss of a failure. This allows the 
method to be applied in situations verying from little or no reliability growth to rapid 
reliability growth. 

An example will help clarify the use of the method. Consider the test results 
included in Table 1. The data displayed in the table is from a test-fix-test program in 
time order sequence. A design fix was applied to the equipment after each failure. As 
such, the actual system reliability does not remain constant. The outcome of each trial 
was generated from the geometric distribution using the actual reliability listed in the 
table. The cause for each failure was generated using a stochastic process and has 
been coded using letters. No more information concerning failure cause is required 
because the data are used to determine if the current failure cause has occurred 
previously. 

To. apply Straight Percent Discounting the two input parameters must be 
determined. For the example 
N = 3 
F = .25 

will be used. The Straight Percent Discounting Method is applied sequentially, failure 
by failure, recomputing M, the number of successful trials since last failure, for each 
failure cause as the next data entry is included. At the end of each testing phase, the 
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TABLE 1 

SAMPLE TEST RESULTS 



TRIAL 


FAILURE 


FAILURE CAUSE 




NUMBER 


NUMBER 


A 


B 


c 


D 


RELIABILITY 


1 




S 


S 


s 


S 


.358 


2 


1 


F 


S 


s 


S 


.358 


j 


2 


S 


F 


s 


S 


.412 


4 


3 


S 


S 


F 


S 


.518 


5 


4 


s 


S 


s 


F 


.596 


6 




s 


S 


s 


S 


.627 


7 


5 


s 


S 


F 


S 


.627 


8 




s 


S 


S 


S 


.721 


9 




. s 


S 


S 


S 


.721 


10 


6 


s 


F 


S 


S 


.721 


11 




s 


S 


S 


S 


.780 


12 




s 


S 


S 


S 


.780 


13 




s 


s 


S 


s 


.780 


14 




s 


s 


s 


s 


.780 


15 




s 


s 


s 


s 


.780 


16 




s 


s 


s 


s 


.780 


17 




s 


s 


s 


s 


.780 


18 


7 


F 


s 


s 


s 


.780 



S : DENOTES SUCCESS 
F : DENOTES FAILURE 



value of M, along with the input parameters F and N, is used in Equation 2.3 to 
determine the adjusted failure value. This procedure is applied to the data in Table 1. 

When the sixth failure is included, all failures due to Causes A, C or D have 
values for M greater than or equal to three causing the discount fraction to be applied. 
Computation of the adjusted failure values for Cause A is illustrated in the following 
calculations. 
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ADJUSTED FAILURE = (1 - .25) INT < 8 '' 3 ) 

= ( -75 ) 2 
= .5625 

Failure two and failure six were both due to Cause B which resets their corresponding 
M values to zero and their adjusted failure value to unity. The results for all failures to 
date are shown below. 



FAILURE 




FAILURE 


ADJUSTED 


NUMBER 


M 


CAUSE 


FAILURE 


1 


8 


CAUSE A 


.5625 


2 


0 


CAUSE B 


1.0000 


0 


J 


CAUSE C 


.7500 


4 


5 


CAUSE D 


.7500 


5 


0 


CAUSE C 


.7500 


6 


0 


CAUSE B 


1.0000 



Finally, failure seven is included and the failure discounting computations are 
completed. The number of trials, eight, causes further discounting for all previous 
failures except for the first failure which is also due to Cause A. The final results of 
applying the straight percent discount method to the data in Table 1 appears in Table 
2. The values in the adjusted failure column are the current values of each fractional 
failure after trial number eighteen has been included. For example, the two adjusted 
failure values due to Cause B have recorded eight successfull trials since the last failure 
due to Cause B and INT (8/3) = 2. Therefore, the adjusted failure value equals 
(.75) = .5625. The remaining adjusted failures are computed in the same manner 
yielding the fractional failure values after-eighteen trials and seven failures. 

2. Lloyd Failure Discounting Method 

The underlying premise of the Lloyd Discounting Method states that the 
fraction of a failure removed by the discounting method should not be arbitrarily 
chosen. Some statistical basis should be used to determine how much, if any, a past 
failure should be discounted. Lloyd proposes using the upper confidence limit for the 
probability that a failure cause will reoccur as the discounted value of its corresponding 
failure. This technique allows the user to control the amount failures are discounted by 
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TABLE 2 










STRAIGHT PERCENT DISCOUNTING 
OF EXAMPLE DATA ( TABLE 1 ) 


TRIAL 


FAILURE 


FAILURE CAUSE 


ADJUSTED 


NUMBER 


NUMBER 


A 


B 


c 


D 


FAILURE 


1 




S 


S 


s 


S 




9 


1 


F 


S 


s 


S 


1.0000 


3 


2 


S 


F 


s 


S 


.5625 


4 


3 


S 


S 


F 


S 


.4219 


5 


4 


s 


S 


S 


F 


.3164 


6 




s 


S 


s 


S 




7 


5 


s 


S 


F 


s 


.4219 


8 




s 


S 


S 


s 




9 




s 


s 


S 


s 




10 


6 


s 


F 


S 


s 


.5625 


11 




s 


s 


S 


s 




12 




s 


s 


S 


s 




13 




s 


s 


s 


s 




14 




s 


s 


s 


s 




15 




s 


s 


s 


s 




16 




s 


s 


s 


s 




17 




s 


s 


s 


s 




18 


7 


F 


s 


s 


s 


1.0000 




S : DENOTES 


SUCCESS 






F : DENOTES FAILURE 





specifying the level of confidence for the confidence bound. As such, a single input 
parameter, the confidence level y, is required to perform failure discounting. (Ref. 2] 
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The discounting equation is derived from the upper confidence limit for the 
probability of failure due to a particular failure cause. This confidence interval is based 
on the number of successful trials recorded since implementation of the "fix" for the 
failure cause. Assuming the "fix" is implemented immediately after failure, this will be 
equivalent to the number of successful trials since last failure, previously defined as M. 

Q = PROBABILITY OF FAILURE IN A SINGLE TRIAL 
= 1 - R 

y = FRACTIONAL CONFIDENCE LEVEL 

M = NUMBER OF SUCCESSFUL TRIALS SINCE LAST FAILURE 
g L(y) = LOWER Y% CONFIDENCE LIMIT FOR RELIABILITY 

-L(y) - 0 - Y) 1/M 

Y = Pr [ R > R l(y) ] 

= Pr[l-R < l-g L(y) ] 

“ Pr[Q * l-fi L(y) ] 

A UPPER CONFIDENCE LIMIT for Q = l - ( 1 - y M (eqn 2.4) 

ADJUSTED FAILURE = I - ( 1 - y )V M for M > 0 (eqn 2.5) 

= 1.0 for M = 0 

The failure discounting equation is derived in Equations 2.4 and 2.5. Because the 
derived equation for adjusted failures is not defined for M equal zero, the adjusted 
failure value is set equal to one where M equals zero. This is the logical choice since no 
discounting should be applied to a failure cause that is the current cause of system 
failure. This also has the advantage of providing a penalty factor that will restore 
fractional failures to unity if a failure cause reoccurs. 

To apply the Lloyd discounting method, the confidence level. y> must be 
specified. The method is then applied sequentially, failure by failure, recomputing M, 
the number of successful trials since last failure, for each observed failure cause as the 
data from each trial is included. At each step, M is used in Equation 2.5 to compute a 
new adjusted failure value for each observed failure. 
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An example will help clarify the use of the Lloyd method. Applying the 
method to the sample data in Table 1 with y equal to .99 yields the following results 
after the second failure is included. 



FAILURE 


FAILURE 


ADJUSTED 


NUMBER 


M CAUSE 


FAILURE 



1 


1 


CAUSE A 


.9900 


2 


0 


CAUSE B 


1.0000 



Including the data for the third failure, due to failure Cause C, both Cause A and B are 
further discounted. The adjusted failure value for Cause A is computed below. 



M = 

ADJUSTED FAILURE = 



2 

1 - ( l - .99 ) 1/2 
1 - V -01 
1 - .1 
.9 



The complete results are shown below. 



FAILURE 




FAILURE 


ADJUSTED 


NUMBER 


M 


CAUSE 


FAILURE 


1 


2 


CAUSE A 


.9000 


2 


1 


CAUSE B 


• .9900 


3 


0 


CAUSE C 


1.0000 



The process continues sequentially, adding the next failure, recomputing M, and 
computing the adjusted failure values for each previous failure. The final results after 
all seven failures are included is shown in Table 3. Note, as a failure cause reoccurs its 
corresponding value for M is set to zero forcing the adjusted failure value to unity. 

This concludes the description of the two discounting methods covered in the 
study. The following section will describe the three discrete reliability growth models. 
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TABLE 3 










LLOYD DISCOUNTING 
OF EXAMPLE DATA ( TABLE 1 ) 




TRIAL 


FAILURE 


FAILURE CAUSE 


ADJUSTED 


NUMBER 


NUMBER 


A 


B C 


D 


FAILURE 


1 




S 


S 


S 


S 




2 


1 


F 


S 


S 


S 


1.0000 




2 


S 


F 


S 


S 


.4377 


4 


3 


S 


S 


F 


s 


.3421 


5 


4 


S 


S 


S 


F 


.2983 


6 




S 


S 


S 


S 




7 


5 


S 


S 


F 


S 


.3421 


8 




S 


S 


S 


S 




9 




S 


S 


S 


S 




10 


6 


S 


F 


S 


S 


.4377 


11 




S 


S 


S 


s 




12 




S 


S 


S 


s 




13 




S 


S 


S 


s 




14 




S 


s 


S 


s 




15 




S 


s 


s 


s 




16 




S 


s 


s 


s 




17 




S 


s 


s 


s 




18 


7 


F 


s 


s 


s 


1.0000 




S : DENOTES SUCCESS 






F : DENOTES FAILURE 





C. RELIABILITY GROWTH MODELS DESCRIPTION 

Of particular interest in this study are test plans which terminate each test phase 
with system failure. Such test plans are known as testing to a fixed number of failures. 
Only one of the three reliability growth models proposed can be used for test scenarios 
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which terminate testing after a fixed number of trials. Because of this limitation, only 
test plans which test to a fixed number of failures are considered in this study. 

Three discrete reliability growth models are presented in this section, each of 
which uses a different technique to track changing system reliability. The first model, 
Maximum Likelihood Estimate With Discounting, is derived from the conventional 
maximum likelihood estimate of reliability. Failure discounting gives the model the 
ability to track changing reliability. Two additional models derived from an 
exponential single phase reliability estimate referred to as the Weighted Average Model 
and the Exponential Regression Model are also discussed. The MLE With 
Discounting model will be presented in the next section. 

1. Maximum Likelihood Estimate With Failure Discounting 

The most commonly used estimate of system reliability is the maximum 
likelihood estimate Equation 2.2 restated below. 

Number of Successful Trials 

R = (eqn 2.2) 

Total Number of Trials 

Where the number of successful trials equals the total number of trials minus the 
number of observed failures. Because all testing considered is conducted to a fixed 

k 

NO. SUCCESSFUL TRIALS = .2 ^ ( T. - 1 ) (eqn 2.6) 

number of failures, the number of successful trials simplifies to Equation 2.6. Where T 
equals the number of trials following the (i— 1) st failure including the i ^ failure and 
k equals the total number of failures in the test program to date. 

Two problems arise when using Equation 2.2 to estimate reliability growth. 
First, the estimate takes a large number of trials to accurately estimate system- 
reliability. Second, the estimator requires a constant reliability, R, for each trial. 
Because the reliability at each phase, R k , may not be constant with k; only the test 
data from the phase of interest may be used to estimate reliability. Failure discounting 
is introduced into the model in an attempt to adjust the underlying reliability for data 
from previous phases, making it compatible with the data from the current phase. This 
increases the amount of data available for estimating reliability at each phase, 
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improving the accuracy of the estimate. Choosing the correct discounting method with 
the correct parameters is critical to this model because the previous test data must be 
properly adjusted to reflect an underlying reliability that is compatible with the current 
phase. This problem is difficult to solve because the actual reliability in each phase is 
unknown- 

introducing failure discounting creates a computation problem because the 
number of failures is no longer an integer value. This problem can be resolved in one 
of two ways. The failure can remain a fractional value or the failure value can be reset 
to unity by adjusting the observed number of trials up to and including failure. Which 
ever method is chosen, the ratio of failures to total trials must remain constant or the 
estimate of reliability will be altered. Returning the failure value to unity and adjusting 
the number of trials is the method chosen to resolve this problem. This solution allows 
the data to retain its original structure of the number of trials up to and including 
failure. 

Adjusting fractional failures and the number of trials up to and including 
failure is accomplished by dividing both values by the fractional failure value. This 
returns the number of failures to unity and adjusts the number of trials such that the 
ratio between the number of failures and the number of trials remains constant. The 
adjusted number of trials may not be an integer value. This does not present a problem 
for computing the MLE. In cases where the total number of trials must also be an 
integer value, the adjusted number of trials is rounded to the nearest integer. This will 
have a minimal effect on the ratio of failures to trials. 

For example the following data is the result of applying a failure discounting 
method to observed test data. 

FRACTIONAL FAILURE = .5 
NUMBER OF TRIALS = 6 

Dividing both values by .5 adjusts the fractional failure to unity and yields an adjusted 
failure value. 

FAILURE VALUE = 1.0 
ADJUSTED NUMBER OF TRIALS = 12 
The ratio of failures to trials remains constant. 

.5/6 = 1/12 = .0833 
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In general, Equation 2.7 is used to adjust the number of trials when the adjusted failure 



ADJUSTED # TRIALS 



OBSERVED # of TRIALS 
ADJUSTED FAILURE 



(eqn 2.7) 



value is returned to unity. 

The reliability growth model is applied to the data sequentially, phase by 
phase. After the test data from a phase is recorded, the discounting routine is applied 
to the data and an adjusted failure value is computed for each observed failure. The 
adjusted number of trials for each observed failure is computed using Equation 2.7. 

(Y ADJUSTED TRIALS) - J 

R. = — ^ (eqn 2.8) 

k Y ADJUSTED TRIALS 

J = TOTAL NUMBER OF FAILURES IN THE TEST PROGRAM TO DATE 

Finally, the estimate of system reliability in phase k is computed using Equation 2.8. 
This procedure is repeated after each testing phase is completed yielding an estimate of 
system reliability for each phase. These estimates can then be used to plot the 
estimated reliability growth pattern. 

To illustrate the use of the model, the discounted failure values from Table 2 
are used to compute an estimate of system reliability. Because each data point 
represents a development phase, the estimate will be of phase seven reliability. The 
results of the computations are shown in Table 4. 

Note, the sum of the adjusted trials is not rounded to an integer value. Using 
only the data in the seventh phase, the MLE without discounting is 
R 7 = 7/8 = .875 . 

The MLE using all test data without discounting will yield 
R 7 = 11/18 = .611 . 

For this example, the MLE with discounting provides an improved estimate of actual 
reliability; i.e. R 7 = .744 is closer to the actual reliability value, .78. 

When testing is terminated at a fixed number of failures, the MLE has an 
additional shortcoming; it is conservatively biased. Breaking the data into sets of trials 
up to and including a single failure, the number of successful trials in a set always 
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TABLE 4 



MLE WITH DISCOUNTING 
OF SAMPLE TEST DATA (TABLE 2) 



PHASE # 


CAUSE 


ADJ. FAILURE 


ADJ. TRIALS 


1 


A 


1.0000 


2.0000 


2 


B 


.5625 


1.7778 


3 


C 


.4219 


2.3702 


4 


D 


.3164 


3.1606 


5 


C 


.4219 


4.7405 


6 


B 


.5625 


r ^ ^ 


7 


A 


1.0000 


8.0000 


1 . FT 

II II 

' “-4 






£ = 27.3824 



R 1 = ( 27.3824 - 7 ) * 27.3824 
= .74436 

R = .78 



equals the number of trials in a set minus one. As such, the MLE. Equation 2.2, may 
be stated as follows for each set of data. 




X = NUMBER OF TRIALS UP TO AND INCLUDING FAILURE 

The expected value of R is derived in Equation 2.9. Table 5 shows the E[R] compared 
to the actual reliability R. Clearly, the maximum likelihood estimate will be very 
conservatively biased for low reliabilities. (Ref. 3: p.34] 
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E[R) = E[(X-1)/X] 

= 1 " E[ 1/X ] 

E[l/X] = (1-R) + (R/2)(l-R) 4 - (R 2 /3)(1~R) 4 - ... 

= 1 - R 4 - (R/2) - (R 2 /2) + (R 2 /3) - (R 3 /3) + ... 

In(l-R) = -R — (R 2 / 2) — (R 3 /3) 

— (1/R)ln(l — R) = 1 + R/2 + R 2 /3 4- ... 

E[ 1/X] = ln(l - R) - ( 1 / R)ln( 1 - R) 

E[R] = 1 + ( 1 / R)ln( 1 - R) - ln(l - R) (eqn 2.9) 





TABLE 5 


MLE EXPECTED VALUE 


R 


E[g] 


.1 


.052 


.3 


.168 


.5 


.307 


.7 


.484 


.8 


.598 


.9 


.744 


.99 


.953 


.999 


.993 



This bias can be offset by the proper choice of a discounting parameter; but, 
the development of a less biased estimator would be advantageous. The Exponential 
Single Phase Estimate, introduced in the next section, demonstrates a smaller 
conservative bias than the MLE when applied to geometric data. 
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2. Exponential Single Phase Reliability Estimate 

The Exponential Single Phase Reliability Estimate was developed for the 
special case of testing until the first failure is encountered. Test data of this form has a 
geometric distribution with the parameter, R, system reliability. As shown in the 
previous section, the maximum likelihood estimate is conservatively biased when 
applied to geometric data. In an attempt to reduce this bias, an estimator of the 
following form was suggested. 

R = 1 - e' A 
e' A = 1 - R 
A = -ln(l - R) 

To complete the development of the estimator, an unbiased estimate. 
for A was sought such that 
E[£] = -ln(l - R) = A 

The estimate is derived as follows. Let X. f equal the number of trials up to and 
including failure for the i ^ set of geometric data and R ; equal the system reliability for 
the i ^ set of data where i = 1, 2, 3, ... , number of failures. Assume that 
Yj = f ( Xj ) is an unbiased estimate of A. Then 
oo 

£f(Xj) Rj Xi “ 1 (1-Rj) = ~ln(l - R.) 
x= 1 

= R. + R. 2 + R. 3 + ... . 

1 1 1 

Solving for f ( Xj ) by equating coefficients of terms with like powers yields 
f( Xj ) - 1 + 1/2 + 1/3 + ... 4- l/(Xj - 1). 

Resulting in Equation 2.10. 

Y. = 1 + 1/2 + ... +1/(X. - 1) for Xj > 2 (eqn 2.10) 

= 0 for Xj = 1 

Substituting Y ; , an unbiased estimate of A, into the reliability equation yields an 
estimate of single phase reliability, Equation 2.11. Equation 2.11 can be used to 
estimate system reliability when testing is performed until first failure. 

As has been shown, Y. is an unbiased estimate of A; but, this does not ensure 
Equation 2.11 will provide an unbiased estimate of system reliability. To resolve this 
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R = 1 - e' Y i 



I 



(eqn 2.11) 



question Monte-Carlo simulation was used to evaluate the properties of the 
Exponential Single Phase Estimate and compare it with the MLE. 

The algorithm described below was used to evaluate the Exponential Single 
Phase Estimate, Equation 2.11. First a uniform (0,1) random number was generated. A 
table look-up algorithm was used to convert the uniform (0,1) random variable to a 
geometric random variable with reliability R. Estimates of system reliability, based on 
the geometric random variable, were computed using both the Exponential Single 
Phase Estimate and the Maximum Likelihood Estimate. The procedure was then 
replicated a large number of times. The running average of the estimates for each 
estimator was updated after each replication. The results of the simulation are shown 
in Table 6 for ten-thousand replications. 



TABLE 6 

ESTIMATE EXPECTED VALUE 
FOR MLE & EXPONENTIAL ESTIMATE 



THEORETICAL RESULTS SIMULATION RESULTS 



R 


. E[MLE] 


MEAN MLE 


MEAN EXP. EST. 


.1 


.052 


.053 


.066 


.2 


T07 


.107 


.132 


.3 


.168 


.170 


.207 


.4 


.234 


.234 


.281 


.5 


.307 


.305 


.361 


.6 


.389 


.388 


.452 


.7 


.484 


.484 


.552 


.8 


.598 


.598 


.664 


.9 


.744 


.744 


.798 


.99 


.953 


.954 


.968 


.999 


.993 


.994 


.996 



28 



The simulation generated mean for the MLE is very close to its theoretically 
derived value verifying the accuracy of the simulation. Comparing the simulation 
generated mean estimate for the Exponential Single Phase Estimator to the actual 
reliability leads to the conclusion that the estimate is conservatively biased, but less 
biased then the MLE. As is the case with the MLE, the amount of bias in the 
Exponential Single Phase Estimate is a function of the actual reliability. 

Constant system reliability is necessary for the Exponential Single Phase 
Estimate to accurately predict reliability. Because of this property, the estimator must 
be modified to accurately predict the changing system reliability inherent in a reliability 
growth pattern. As was the case with the MLE, the modification should make it 
possible to use all available test data when computing a reliability estimate. The 
remaining reliability growth models explore alternative modifications to the 
Exponential Single Phase Estimate. The first modification employs linear regression to 
estimate the parameter A. The second modification involves taking the weighted 
average of the current single phase estimate and the preceding phase reliability estimate 
to produce an estimate of current phase reliability. 

3. Exponential Regression Discrete Reliability Growth Model 

The Exponential Regression Reliability Growth Model was developed by W. 
M. Woods from the Exponential Single Phase Reliability Model [Ref. 4]. The model 
studied here is identical to the original model with the exception of applying failure 
discounting. The model can only be applied in situations where testing is performed to 
a fixed number of failures and must have attribute data. Any failure discounting 
method may be used with the model; but, the discounted failure data must be returned 
to its original form, an integer number of failures and successes. To apply the model, 
one must record the data necessary to apply the chosen failure discounting method, the 
total number of failures and successes, and the number of successful trials between 
each failure. The equation 

R = 1 - e' A 
where 

A = a + p k 

and k denotes the testing phase is used to compute system reliabilitv 

R = 1 - e-(« + P k) 

k = 1,2,3,... 
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Estimating the expression a + Pk at each phase allows the model to track changing 
reliability. The model is further modified to include the possibility of more than one 
failure in a given phase. Let F k equal the total number of failures in the k ^ phase and 
j equals the failure number in phase k such that j = 1, 2, ..., F k< Nj k equals the 
number of trials between the (j — 1) st and the j ^ failure, including the j t * 1 failure, in 
phase k. As in the Exponential Single Phase Estimate, Equation 2.10, still represents 
an unbiased estimate of ( a + P k) for each set of trials Nj k . To avoid confusion with 
the additional subscript, Equation 2.10 becomes Equation 2.12 



Yj k = 1 + 1/2 4- ... + l/(Xj - 1) 
= 0 



for X ; > 2 (eqn 2.12) 
for X. = 1 



Estimates of a and p for each phase are developed using the techniques of linear 



k 

I (j - K. ) x Y 
i=l 



I (i - K. ) 2 

i-i 



(eqn 2.13) 



regression yielding Equations 2.13 and 2.16. Equation 2.13 requires the computation 
of Y.j , Equation 2.14 , and K. , Equation 2.15. 

A - < Y lk + Y 2k + - + Y F k k )/ p k <«!" 2 - 14 > 



K. = (1 -r 2 + ... + k)/k 



(eqn 2. 15) 



% - Y • - K K - 



(eqn 2.16) 
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Y.. - (Y.j + Y. 2 + ... + Y. k )/k 



(eqn 2.17) 



Substituting the estimates for <* k and P k into the reliability equation yields the estimate 
of reliability for phase k, Equation 2.18. 

# k = 1 — e‘^ a k " r Pk k > 1 (eqn 2.18) 

= 1 - e‘^-l k = 1 

A separate definition of the case where k= 1 is necessary because regression techniques 
require a minimum of two data points. [Ref. 3: pp. 1-4] 

The only modification made to the original model is the addition of failure 
discounting. After each phase, the chosen failure discounting method is applied to the 
test data. The fractional failures that result must be adjusted to unity by adjusting the 
number of trials, Equation 2.7. Computation of Yj k requires the total number of trials 
be returned to an integer value. Therefore, the adjusted trials are rounded to the 
nearest integer. Estimates for p, Equation 2.13, and a, Equation 2.16, are then 
computed. These values are substituted into Equation 2.18 to obtain the estimate of 
system reliability for phase k. 

Because the parameters a and p are computed using linear regression, the 
model has the ability to track changing reliability even if no failure discounting is 
applied. Even so, it may take numerous phases and a great deal of data for the 
Exponential Regression Model to accurately predict reliability. Adding failure 
discounting to the model should decrease the amount of data required for an accurate 
estimate of system reliability. 

The Exponential Regression Model can predict a negative system reliability. 
This occurs most often in the initial testing phases and at low reliabilities. Should this 
occur, the estimate should be set to zero. 

An example will help clarify the application of the model. Using the 
discounted test data from Table 2, the estimate of phase seven reliability is computed 
in Table 7. 

Note the Yj k column was deleted because each phase consists of a single 
failure and Yj k = Y. k< This example also demonstrates the results of applying 
improper discounting parameters for the situation. Failure discounting caused the 
estimator to further overestimate the actual reliability. Had no discounting been 
applied, the regression estimate would have yielded R 7 = .836. 
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TABLE 7 

EXPONENTIAL REGRESSION ESTIMATE COMPUTATIONS 



k 


ADJ. TRIALS 


Y - k 


(k-K.)Y. k 


(k-K.) 2 


1 


2 

Mm 


1.0 


-3.0 


9 


2 


0 

Mm 


1.0 


-2.0 


4 


3 


2 


1.0 


-1.0 


1 


4 


j 


1.5 


0.0 


0 


5 


5 


2.083 


2.083 


1 


6 


5 


2.083 


4.166 


4 


7 


8 


2.593 


7.779 


9 




y 


11.259 


8.023 


28 



K. = 4.0 

Y.. = 11.259/7 = 1.608 

P 7 = (8.028 * 28) = .287 

a ? = 1.608 - (.287 x 4) = .460 

(a 7 + p 7 K) = .460 + (.287 x 7) = 2.469 

R 7 = 1 - exp( -2.469 ) = .915 

R 7 = .78 



The last remaining reliability growth model is also derived from the 
Exponential Single Phase Estimate. A weighted average technique is used to allow the 
estimator to track changing reliability. This estimator is described in the next section. 

4. Weighted Average Discrete Reliability Growth Model 

The Weighted Average Model tracks changing reliability by taking a weighted . 
average of the two most recent reliability estimates. This model is only applicable to 
situations when testing is performed until a fixed number of failures have been 
observed and attribute data is available. A slightly modified version of the Exponential 
Single Phase Reliability Estimate is used in the model. The current phase reliability is 
estimated using a weighted average of the previous phase reliability estimate and the 
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current single phase estimate. The previous phase estimate and the current single phase 
estimate are weighted by the fraction of total trials to date used in the computation of 
each estimate. As with the Exponential Regression Estimate, this model will track 
changing reliability without the use of failure discounting; but, like most weighted 
averages, the model may lag actual growth by one or more phases. Failure discounting 
is used to correct the lag and reduce the number of trials necessary to compute an 
accurate estimate. 

Equation 2.11 is applicable when testing to first failure. A slightly different 
version of the estimator is applicable to situations where more than one failure has 
been recorded. To apply this estimator, a running total of the number of successful 
trials to date, S k , and failures to date, F k , is maintained for each phase k. These totals 
are used in Equation 2.19 to compute the current single phase reliability estimate, 

- $ Pk 

5 S p k = 1 ~ e'-k (eqn 2.19) 

£ k = l//F k l ^ F k + !) + — + V[(F k + S k ) - 1 ] 



For this model, running totals of failure and success data are recorded for 
each observed failure cause. As each trial is performed, the number of successes for 
each failure cause, except the current failure cause, is incremented by one. Should a 
previously recorded failure cause reoccur, its failure total is incremented by one. The 
system values for F k and S k are then selected from the individual failure cause having 
the largest number of failures to date. 

Once the single phase estimate is computed; the estimate of system reliability 
is computed by taking the weighted average of the previous phase reliability estimate 



*k = 



N, 



N 



-k-1 



+ 



N2_ 

N 



-sp k 



k= 2,3," 



(eqn 2.20) 



-sp k k= 1 

and the current single phase estimate, Equation 2.20. 

Where N equals the total number of trials recorded to date. N 1 equals the total 
number of trials recorded in previous phases. N 2 equals the number of trials recorded 
in phase k. Note the sum of Nj and N 2 equals N. 
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Failure discounting is applied to the model in much the same way as it was in 
the Exponential Regression Model. After a discounting method has been applied to 
the data, fractional failures are returned to unity by adjusting the number of trials, 
Equation 2.7. The adjusted number of trials is then rounded to the nearest integer. It is 
important to note, when failure discounting is applied to this model S k must be 
recomputed from phase one at each phase boundary because the recorded number of 
successful trials for each previous phase may be altered by the discounting routine. 

An example of the model's application should help clarify the required 
computations. Using the discounted failure data from the example of the Straight 
Percent Discounting Method leading up to Table 2, at the end of phase one the sum of 
failures and successes for the failure cause is listed below. 

FAILURES SUCCESSES 
CAUSE A 1 1 

Solving for the single phase estimate and the reliability estimate yield 

c x = 1.0 

Si - S SP] - 1 - e- 1 = .6321 . 

After the second phase data is included and the failure discounting method applied, the 
sum of failures and successes for each failure cause is listed below. 



FAILURES SUCCESSES 
CAUSE A 12 

CAUSE B' 1 2 



From the failure data, F 2 equals one and S 2 equals two yielding C 2 equal to 1.5. 
Applying the remaining model yields the second phase reliability estimate R . ,. 

= 1 - e'< 3/2) = .7769 

Nl = 2 

n 2 = 1 

N = 3 



R 2 = (2/3)^ + (1/3) * sp ^ 

= (2/3)(.632 1) + (1/3)07769) 
= .6804 
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The computations are repeated for each phase of the testing program. As 
mentioned previously, when failure discounting alters the number of successful trials, 
S k , in a previous phase, the reliability estimates for all previous phases must be 
recomputed to solve for the current phase reliability estimate. As this simple example 
clearly shows, this reliability growth model is computationally intense. Programming 
the model on a computer may be necessary to compute reliability estimates for large 
data sets. 

Two failure discounting methods and three discrete reliability growth models 
have been described in this chapter. Evaluation of each model is very difficult because 
the expected value and variance of the estimates, when discounting is applied, become 
mathematically untractable. It is also not possible to evaluate the models using actual 
test data because the true reliability at each phase is unknown. The following chapter 
discusses a Monte-Carlo simulation written to evaluate the performance of each of the 
failure discounting methods and discrete reliability growth models. 



35 



III. DISCRETE RELIABILITY GROWTH SIMULATION 



Evaluating reliability growth estimators including reliability growth models, is 
often difficult. True system reliability is unknown, rendering actual test data useless for 
evaluating an estimators performance. When the mathematics is tractable, the 
properties of the estimator can be derived and used to evaluate the estimators 
performance. However, many estimators are too complex to allow mathematical 
solutions for the closed form of their properties. For this group of estimators, 
computer simulation is one of the only evaluation techniques available. The addition of 
failure discounting to an estimator makes mathematical solutions very difficult if not 
impossible. The reliability growth models introduced in this study fall in the category 
requiring simulation as an evaluation tool. 

A Monte-Carlo simulation was written to evaluate the properties of the three 
reliability growth models introduced in Chapter II. The simulation builds a reliability 
growth pattern and generates test data from the known system reliabilies using the 
geometric distribution. Each of the reliability growth models are then applied to the 
simulation generated test data. Repeating the process many times for a particular set of 
known reliabilities will provide a good estimate of both the mean and variance of each 
estimator as well as provide information about the underlying distribution of the 
estimate. These statistics can then be used to evaluate each reliability growth model's 
performance. 

Several modeling problems had to be solved before the simulation could be built. 
A method for generating a realistic reliability growth pattern had to be developed. This 
method must closely model the process of testing equipment, implementing design 
improvements or "fixes" and continued testing. It also must providing the flexibility to 
test numerous reliability growth patterns. Once the actual system reliability in ’each 
phase is established from the growth pattern, test data of the form the number of trials 
up to and including failure must be generated. Because such data are geometrically 
distributed when R is fixed, this process is relatively simple. A relistic method must also 
be developed for determining the cause of each failure. This step proved to be the 
most difficult. 
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As with all simulations, several modeling assumptions were necessary. The 
following section discusses the assumptions and their impact on the simulation. The 
remainder of the chapter describes the development of the simulation algorithm and its 
capabilities and limitations. 

A. MODELING ASSUMPTIONS 

Building an algorithm that will generate realistic test data requires several 
simplifying assumptions about the actual test process. These assumptions can be 
broken into three categories. They are: 

1. General assumptions 

2. Assumptions concerning the true reliability growth pattern 

3. Assumptions concerning failure cause determination 

The last two categories correspond to the two steps in generating test data, generating 
the actual reliability growth pattern and computing test data from the growth pattern. 
Each assumption is listed and discussed by category. 

1. General Assumptions 

The general assumptions narrow the scope of testing considered to those 
scenarios which apply to the reliability growth models evaluated. The two required 
assumptions are: 

1. All testing is performed to a fixed number of failures. 

2. All test results are recorded as attribute data. 

These assumptions ensure the test data produced by the simulation will be compatible 
with the discrete reliability growth models developed in the study. The models can only 
be applied to discrete or attribute data where testing is performed until a fixed number 
of failures are observed. 

2. Reliability Growth Pattern Assumptions 

Defining the mechanisms which cause an items' reliability to change from 
phase to phase is necessary before an algorithm to generate a realistic reliability 
growth pattern can be written. Several assumptions were necessary to properly define 
these mechanisms. They are: 

1. The reliability growth pattern is non-decreasing. 

2. System reliability changes only at phase boundaries. 

3. Equipment improvements are implemented immediately after a phase ends and 
before any further testing. 
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4. Only design weaknesses causing failure during a phase are corrected at the end 
of a phase. 

5. Each design improvement removes a fixed fraction of the probability of 
reoccurrence for the corresponding failure cause. 

The first assumption limits the shape of the generated growth patterns to non- 
decreasing curves. Because one of the objectives of development testing is to improve 
system reliability, this assumption is logical. However, actual reliability growth may 
have periods of decline when "fixes" to design problems actually decrease reliability. 
Over the entire testing period, even these curves are generally increasing. The worst 
case this assumption allows is no growth over the testing phases. This should be 
adequate for determining adverse growth characteristics of the reliability growth 
models. 

Assuming system reliability remains constant throughout a testing phase 
makes the model consistent with the previous definition of a testing phase. This 
assumption means no design changes or any other improvements are made to an item 
on test within a phase. Because a phase can be defined as any number of failures 
greater than or equal to one, this assumption should have little impact upon the 
accuracy of the simulation. 

Assuming all fixes are implemented after a test phase ends and before the next 
phase begins is closely related to the previous assumption. This assumption does 
preclude accurate modeling of long term fixes whose implementation are postponed 
while testing continues. The impact of this limitation should be minimal. 

Assumption four asserts that the only known weaknesses in system design are 
those which have caused system failure. This is not always the case. Suspected 
weaknesses may be corrected along with those weaknesses confirmed by system failure. 
The simulation is unable to model this situation. Even though this deviates somewhat 
from reality, most major improvements to system reliability should come from failure 
identified weaknesses. The intent of the simulation is only to generate a realistic 
reliability growth pattern not to duplicate an actual growth pattern. Therefore the 
simulation results should be adequate to accomplish the task for which the model was 
designed. 

The last assumption in this category deals with the mechanism used by the 
simulation to model improvements in system design. It asserts that all fixes remove a 
constant fraction of the probability of occurrence for the failure cause of interest. In 
reality many factors determine how effective a particular fix will be and all fixes are not 
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equally effective. This method also decreases the probability of occurrence more for a 
failure cause with a high probability of occurrence than for a failure cause with a low 
probability of occurrence. This also is unrealistic. Despite the shortcomings of this 
simplification, it does provide a simple way to model reliability improvement due to 
design change and wall produce a growth pattern that is adequate for evaluating the 
reliability growth models studied. 

The final category of assumptions deal with generating test data from a known 
system reliability and determining the cause of each failure. They are discussed in the 
next section. 

3. Failure Cause Determination Assumptions 

Realistically modeling failure causes is critical to the evaluation of the failure 
discounting methods. The technique used to model failure cause determination must 
also be incorporated into the technique used to model reliability growth because the 
two processes are closely related. The assumptions made in modeling failure cause 
determination are: 

1. There exists a finite number of possible failure causes. 

2. Each failure cause has a fixed probability of occurrence in each phase. 

3. System reliability can be modeled as a series system of the failure causes. 

4. Each failure cause is stochastically independent of the other failure causes. 

Assuming a fixed number of failure causes may be an oversimplification of 
reality but should not invalidate the model. Even though the possible number of failure 
causes is most likely infinite, only a finite number of these causes with relatively high 
probabilities of occurrence are of interest. The number of possible failure causes 
considered by the simulation must be specified as an input to the model; but, the 
number of causes input is only limited by the capacity of the computer on which the 
model is run. Therefore, the simulation is flexible enough to model nearly any scenario 
of interest. 

Fixing the probability of occurrence for all failures causes within a single 
phase is a direct result of fixing the system reliability for all trials in a single phase. 
This assumption will not further restrict the ability of the algorithm to model reality. 

An assumption is needed to interrelate failure cause determination and system 
reliability growth through design improvements. Assuming the component being tested 
may be modeled as a series system of failure causes meets this requirement. System 
reliability at any phase is a function of the probability of occurrence for the specified 
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failure causes. As such, implementing a design improvement reduces the probability of 
occurrence for a failure cause which in turn increases the systems reliability. This 
assumption also makes it possible to stochastically determine the cause of failure. The 
mathematical derivation of this technique 'will be presented in a later section. 
Modeling the failure cause as a series system does not depart far from reality because 
the occurrence of a failure cause implies system failure. Once the assumption of a 
finite set of failure causes is accepted, modeling them as a series system does not 
further restrict the models representation of reality. 

The final assumption requires that each failure cause be stochastically 
independent. This means that the causes of failure are not related. The occurrence of 
one cause does does not influence the probability of occurrence for any other cause. If 
the failure causes are properly defined, this assumption should not limit the accuracy of 
the model. 

The following section discusses the simulation algorithm built from these 
assumptions. 

B. SIMULATION ALGORITHM DEVELOPMENT 

The algorithm developed for this simulation can be broken into the four sections. 
They are: 

1. Generate a reliability growth pattern. 

2. Generate test data, including the cause of failure, from the reliability growth 
pattern. 

3. Apply the failure discounting methods and reliability growth models to the 
simulation generated data. 

4. Compute statistics used to evaluate the performance of each model. 

The first two steps of the algorithm along with the output- statistics will be 
discussed in detail in this section. The methodology used to apply each of the 
discounting methods and reliability growth models was discussed in Chapter II and will 
receive little attention here. With few exceptions, the actual content of the Fortran 
code used in the simulation will not be discussed. A documented copy of the code is 
included at Appendix A for the interested reader. 

1. Simulation of a Reliability Growth Pattern 

The reliability growth pattern built by the simulation models the actual 
process of testing and improving an item in a development program. Several user 
inputs are required at this stage of the simulation. They are: 

1. number of testing phases 
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2. number of failures in each testing phase 

3. number of possible failure causes 

4. the first phase probability of occurrence for each possible failure cause 

5 . growth fraction (F a ) 

o 

Once these parameters are in place, the simulation computes the first phase 
system reliability from the probability of occurrence for each possible failure cause. The 
number of trials up to and including failure and the failure cause are then computed for 
all failures in the first phase. The algorithm used, to generate system reliability and the 
failure data will be covered in the next section. 

After the failure data for the first phase has been generated, the probability of 
occurrence for each failure cause which occurred in the first phase is recomputed using 
the growth fraction. Assume the first failure occurred due to cause A, the new 
probability of occurrence, Pr (occurrence) n , will be computed using Equation 3.1. 

Pr (occurrence^ = Pr( occurrence) x (1 — F ) (eqn 3.1) 

This equation is applied at the end of each phase to those failure causes that occurred 
in the current phase. If a failure cause is responsible for multiple failures in the phase, 
Equation 3.1 is applied only one time. This procedure simulates identifying a system 
weakness through testing and implementing a design improvement. The procedure is 
repeated at the end of each testing phase. 

The result of the algorithm is a stochastically determined reliability growth 
pattern that simulates actual developmental testing. A test is performed, weaknesses in 
the system are identified, corrective design changes are implemented and testing is 
continued. This algorithm also has the advantage of interrelating the probabilities of 
occurrence for each failure cause and the system reliability. This will be useful in 
determining the failure cause. By changing the growth fraction, the rate of reliability 
growth can be varied from no growth to rapid growth. 

2. Generating Test Data From Known Reliability 

Two problems must be solved before an algorithm that generates test data can 
be developed. The system reliability must be determined from the probability of 
occurrence for each failure cause. A technique for generating the number of trials up to 
and including failure from known reliability that identifies the failure cause also must 
be developed. The solution to both problems lies with the assumption that system 
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reliability can be modeled as a series system of the failure causes. In a discrete series 
system the number of trials up to and including system failure will be the minimum of 
the number of trials up to and including failure for each sub-component. Let M equals 
the number of trials up to and including failure for the system and Xj where 
i= 1 ,2 , ... , j denotes the number of trials up to and including failure for the i U1 
subcomponent, here a failure cause. If the j components form a series system, M may 
be expressed as 

M = MIN (X 1 ,X 2 ,...,X j ) 

Because the probability of occurrence is fixed throughout a phase for each failure 
cause, Xj is distributed as a geometric random variable with parameter it. From an 
assumption, the X ; 's are also mutually independent. It can be shown for a geometric 
random variable, Xj, that 
Pr (X ; > n) = R". 

The distribution for M is derived as follows. 

M = MIN[X 1( X 2 Xj] 

Pr [M > n] = Pr [ Xj > n, X 2 > n, ... ,Xj > n ] 

= Pr [Xj > n] x p r [X 2 > n] x ... x p r [Xj > n] 

= Rj n x R 2 n x ... x R. n 
= [ Rj x R 2 x ... x R. ] n 

Therefore M is also distributed as a geometric random variable with the parameter 

R = ( Rj x R 2 x ... x R. ) . 

The system reliability at each phase is computed as the product of the 
compliments of the probabilities of occurrence for each possible failure cause. Because 
of this computation the actual simulation input is the compliment of the first phase 
probability of occurrence for each failure cause. As the probability of occurrence for 
each failure cause is recomputed from phase to phase, the system reliability is also 
adjusted. 

The number of trials up to and including failure for a geometrically distributed 
random variable with reliability R can be generated from a uniform (0,1) random 
variable, U, using Equation 3.2. [Ref. 5]. 

X = INT [ 1 + ( ln(U)/ln(R) ) ] (eqn 3.2) 
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Equation 3.2 combined with the definition of a series system provide the solution to 
generating the number of trials up to and including failure while accounting for the 
cause of failure. 

For each failure generated by the simulation, Equation 3.2 is used to generate 
a geometric random variable for each failure cause using the compliment - of the 
probability of occurrence as the parameter. The minimum geometric random variable 
over the failure causes becomes the number of trials up to and including failure for the 
system. The cause of system failure is recorded as the failure cause producing the 
minimum geometric random variable. Generating the test data in this fashion allows 
the reliability growth pattern and test data to be stochastically generated yet 
interrelated much as they are in nature. A summary of the steps required to compute 
the reliability growth pattern and the test data follows. 

1. Input the required data 

2. Generate phase failure data including the number of trials and the failure cause. 

3. Recompute the probability of occurrence for the failure causes that occur in the 
phase. 

4. Return to step 2 until all phases are complete. 

Once the test data has been computed from the known reliability in each 
phase, the failure discounting methods and reliability growth models can be applied to 
the test data. The resulting reliability estimates can then be evaluated against the 
known system reliability. 

3. Application of the Discounting Methods and Reliability Growth Models 

The discounting methods and reliability growth models are applied to the 
generated test data in much the same way as they were described in Chapter II. Each 
of the three reliability growth models are applied to each set of test results. However, 
the. simulation user must specify one of the two failure discounting methods because 
the simulation can not apply both simultaneously. The maximum likelihood estimate 
without discounting is also computed for each phase, independent of all other test data. 
This allows a comparison between the standard reliability estimator and the reliability 
growth .models. The simulation produces an estimate of system reliability from each 
reliability growth model for each testing phase. The single phase MLE and actual 
system reliability are provided as a basis for comparison. 

One minor departure from the application of the reliability growth models as 
described in Chapter II was necessary. For large numbers of trials up to and including 
failure, N, computations of the form 
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Y - 1 + (1/2) + ... + [1/(N- 1)] 

required extensive computer processing time to solve recursively. To improve program 
efficiency, the values for Y up to and including one-thousand were computed exactly 
and stored in an array. For values of N greater than one-thousand non-linear 
regression was used to estimate the value of Y. Equation 3.3 provided an adequate 
estimate with an R 2 value of .99635. 



Y = 3.55445 x N'C 11 / 100 ) 



(eqn 3.3) 




Figure 3.1 Regression Estimate of Y. 

Figure 3.1 compares the actual Y value with the regression predicted value for large N. 
Both the Weighted Average Model and the Exponential Regression Model are affected 
by the approximation. The approximation has minimal impact upon the computation 
of model estimates. Values for N above one-thousand are rare even at high reliability 
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and values above 15,000, the accurate extension of the approximation, are very rare. 
Should a value of N larger than 15,000 occur, some error will be introduced into the 
model estimate, and this error will increase with N. Because these values are rare, this 
error is not considered significant. 

Further research, after the simulation was written, has yielded a more accurate 

Y = C + log e (N)+l/(2N)-l/[12N(N+l)]-l/[12N(N+l)(N + 2)] (eqn 3.4) 

C = Euler's Number = 0.5772156648 

approximation for Y. This series solution is shown in Equation 3.4 [Ref. 6]. This 
equation should be incorporated into the simulation to improve accuracy in future 
runs. 

The estimates resulting from the application of the reliability growth models 
are retained through the desired number of replications to allow the computation of 
descriptive statistics. These statistics aid in evaluating each growth model. 

4. Statistical Summary of Model Results 

Several descriptive statistics are generated by the model that aid in evaluating 
the performance of each reliability growth model. They include the mean and standard 
deviation of the reliability estimates at each phase for each of the growth models. A 
confidence interval is also computed for the mean estimate at each phase. 

Replications of the simulation are required to compute the descriptive 
statistics. After the first replication, the actual system reliability in each phase is fixed. 
At each replication, new test data is generated from the fixed reliability growth pattern 
and each of the reliability growth models is applied to the data. The estimates 
generated by the growth models are retained at each replication. When the desired 
number of replications have been completed, the sample mean and sample standard 
deviation for each estimator are computed using standard formulas. A 95% confidence 
interval for the mean estimate is computed for each phase reliability estimate with the 
assumption that each estimate is normally distributed. The assumption of normality is 
not valid in every case but the confidence interval is included as a tool for comparison. 
Further analysis of the reliability growth models can be accomplished by saving each 
estimate in an output file. The models user can then analyze the data as desired. 

Changing the simulations' input parameters makes it possible to analyse a 
wide variety of testing situations. These capabilities are discussed in the following 
section. 
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C. CAPABILITIES OF THE MODEL 

Within the limits of the assumptions stated earlier, the simulation is very flexible. 
Any number of failure causes may be input into the model. By varying the phase one 
probability of success for each failure cause, the growth fraction and the number of 
phases a wide range of reliability growth curves may be explored. The random number 
generator seed is also user specified allowing the exact string of random numbers and 
test data to be duplicated. In this way, various discounting methods and parameters 
mav be evaluated using the exact same data. 

The test-fix-test scenario may be modeled by specifying a single failure per phase. 
The test-find-test methodology may be simulated by specifying multiple failures in each 
phase. There is no requirement that each phase have an equal number of failures. 
Therefore a mixed scenario, test-fix-test and test-find-test, may be simulated. 

Each of the discounting methods described in Chapter II may be applied to the 
data using the full range of input parameters. Proper parameter selection for the 
discounting method will cause no discounting to occur. In this manner each of the 
reliability growth models may be evaluated without failure discounting applied. 

The simulation provides a tool for evaluating the reliability growth models 
against a known system reliability growth pattern. When large numbers of replications 
are run, information concerning the expected value and variance of each estimator may 
be obtained. The combinations of failure discounting parameters, reliability growth 
models and anticipated growth patterns may also be evaluated. The following chapter 
discusses the results of numerous runs of the simulation with varying growth patterns 
and failure discounting parameters. 
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IV. EVALUATION OF RELIABILITY GROWTH MODELS 



The discounting methods and reliability growth models were evaluated using the 
Monte-Carlo simulation described in Chapter III. The results from numerous runs of 
the simulation with varying reliability growth patterns, failure discounting methods and 
failure discounting parameters were reduced to graphical output for evaluation. 

The purpose of the evaluation was to determine general characteristics of each 
reliability growth model. The number of simulation runs was not adequate to prove 
that conclusions reached here will hold for all cases. Also, no fast rules for model 
selection or failure discounting parameter selection were developed. Still, many 
characteristics of the different reliability growth models were apparent from the 
simulation runs completed. In particular the answers to the following questions were 
sought. 

1. How well does the reliability growth model track the actual reliability growth 
pattern? 

2. How did the choice of failure discounting parameters affect the estimates of 
reliability? 

3. How variable were the estimates provided by each of the models? 

It was impossible to evaluate all possible reliability growth patterns that can be 
produced by the simulation. Five widely different reliability growth patterns were 
chosen to be representative of the possible range. Constant reliability cases were 
included to identify models that predict reliability growth, were in fact, none exists. The 
five reliability growth patterns chosen are described below. 

1. low reliability with no growth 

2. high reliability with no growth 

3. low initial reliability with moderate growth 

4. moderate initial reliability with moderate growth 

5. low initial reliability with rapid growth 

All simulation input parameters were kept fixed to insure the different simulation 
runs could be compared. The number of phases was held constant at ten with one 
failure per phase. Five failure causes were used and each simulation was replicated one- 
hundred times. The seed for the random number generator was kept constant for all 
simulation runs insuring the same string of random numbers and, therefore, identical 
test data was generated for each growth pattern. 
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Each reliability growth model is evaluated separately in the following sections. 
The graphical results of the remaining simulation runs are included in Appendix B. 

A. GENERAL RESULTS 

Characteristics common to all the reliability growth models are stated here for 
convenience. 

The variability of each of the estimators decreased with increasing phases. This is 
due to the increased data available to the model with each additional phase. 

The accuracy of each estimate increased and the variability of each estimate 
decreased with increasing reliability. This is a function of testing until a fixed number 
of failures have been observed. As reliability increases, the average number of trials 
required to produce failure will also increase. Therefore, as reliability increases, more 
data is available for the model improving the accuracy of the estimate and decreasing 
the estimate's variability. 

B. WEIGHTED AVERAGE RELIABILITY GROWTH MODEL 

The Weighted Average Reliability Growth Model proved to be ineffective as an 
estimator of changing reliability. The model consistently overestimated the actual 
reliability even when no failure discounting was applied. Figure 4.1 plots the mean 
estimate from each model at each phase. Clearly the mean phase estimates for the 
Weighted Average Model overestimated the actual reliability at each phase. The 
addition of failure discounting only amplified the tendency to overestimate as seen in 
Figure 4.2. Even though the variance of the estimates decreased rapidly with 
increasing phases, the model is not useful in its current form. Therefore, the Weighted 
Average Model was not considered in any further analysis. 

C. MAXIMUM LIKELIHOOD ESTIMATE WITH FAILURE DISCOUNTING 
The MLE With Discounting Model accurately tracked a wide range of reliability 

growth patterns. With proper selection of the failure discounting method and 
parameter, this model tracked each of the reliability growth patterns tested. Figures 
4.2, 4.3, 4.4, 4.5, and 4.6 display the model's performance for each of the five reliability 
growth patterns tested. The model is very sensitive to the selection of the failure 
discounting method and parameter. In general, the model underestimates actual 
reliability in the early phases and slowly converges to the actual value with increased 
test data. 
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Figure 4.1 Reliability Growth Models 
Low Constant Reliability 
No Discounting Applied. 

The choice of failure discounting method and parameter is critical to the MLE 
With Discounting Model. Because the Lloyd Discounting Method does not have a 
discounting interval, it generally reduces the discounted failure value more rapidly than 
the Straight Percent Discounting Method. This results in rapid growth of the reliability 
estimates. The Standard Discounting Method with the discount interval set at one and 
the proper choice of the discount fraction will provide nearly identical results to the 
Lloyd method. For all the reliability growth patterns tested, except low initial reliability 
with rapid growth, the Lloyd discounting Method caused the model to overestimate the 
actual reliability. The overestimation was particularly pronounced for low constant 
reliability where the Lloyd Discounting Method forced the model to overestimate 
reliability even with y as large as .99999, Figure 4.7. Because of this characteristic, the 
Lloyd discounting Method is not recommended for use with the Maximum Likelihood 
Estimate. 
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Figure 4.2 Reliability Growth Models 
Low Constant Reliability 
Std. Discounting Applied. 

With the addition of the second parameter, the discount interval, the Straight 
Percent Discount Method is more flexible than the Lloyd Method. Even so, it has the 
potential to overestimate the system reliability. The choice of model parameters is 
critical to the model's performance. Figure 4.8 demonstrates the change in model 
estimates with varying discounting parameters. Several factors influence the proper 
choice of parameters. Three of the most important are the amount of testing to be 
performed, estimates of system reliability before testing begins and how rapidly 
reliability is expected to grow throughout the development and testing program. No 
rales have been developed to help select the proper discounting parameters. Experience 
with the model, good engineering judgement and possibly simulation are necessary to 
select a combination of discounting parameters that will cause the model to accurately 
track the actual reliability growth pattern. Even an apparently good choice of failure 
discounting parameters may result in erroneous reliability estimates if the true 
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Figure 4.3 Reliability Growth Models 
High Constant Reliability 
Std. Discounting Applied. 

reliability does not grow as expected. Therefore, when using this model, frequent 
engineering evaluations may be necessary to insure design evolution is progressing as 
expected. 

Reliability estimates from the Maximum Likelihood Estimate With Discounting 
Model exhibited the smallest variance of the models evaluated. Figures 4.9 and 4.10 
are box plots of the one hundred reliability estimates at each testing phase for two of 
the reliability growth patterns. Comparison with other models may be made from the 
additional box plots included at Appendix B. The reliability estimates from all three 
models exhibited large variance in the first few phases of testing. The variance of the 
estimates gradually decreased as more test data became available. With few exceptions, 
the MLE With Discounting exhibited the smallest variability over the entire range of 
phases for each reliability growth pattern tested. This reduced variance was 
particularly pronounced in the early phases and at low reliability. 
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STD DISCOUNTING, N = 10 ; F = .75 
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Figure 4.4 Reliability Growth Models 
Low Reliability Moderate Growth 
Std. Discounting Applied. 

The MLE With Discounting Model can accurately track most reliability growth 
patterns when the Straight Percent Discounting Method is used. It does so with equal 
or less variance than the other models evaluated. However, the model is very sensitive 
to the choice of of failure discounting parameters and may either over or under 
estimate the actual reliability if the parameters are not chosen correctly. 

D. EXPONENTIAL REGRESSION MODEL 

The Exponential Regression Model has the ability to accurately track most 
reliability growth patterns.. Because it uses a combination of linear regression 
techniques and failure discounting to track changing reliability, it reacts quickly to 
changes in actual reliability. Figures 4.2, 4.3, 4.4, 4.5, and 4.6 demonstrate the model's 
performance with each growth pattern. It also is much less sensitive to the selection of 
failure discounting parameters than is the MLE With Discounting Model. However, 
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Figure 4.5 Reliability Growth Models 
High Reliability Moderate Growth 
Std. Discounting Applied. 

the variance of the early phase estimates is much greater than the estimates from the 
MLE With Discounting Model. 

This model requires several phases, usually at least four, to accurately estimate 
system reliability. The number of phases required is a function of the shape of the 
reliability growth pattern. More phases are required for an accurate estimate at low 
system reliability than at high reliability. The model also appears to track changing 
reliability more accurately than constant reliability. 

The Exponential Regression Model is relatively robust with respect to the 
selection of a failure discounting method and parameters. Still, as was the case with the 
MLE With Discounting Model, the Lloyd Discounting Method tends to cause the 
model to overestimate actual reliability as is demonstrated by Figure 4.11. Figure 4.12 
demonstrates how robust the model can be to changing parameters when the Straight 
Percent Discount Method is applied. Because the Straight Percent Discounting 
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Figure 4.6 Reliability Growth Models 
Low Reliability Rapid Growth 
Std. Discounting Applied. 

Method can produce nearly equivalent results and is much more flexible, use of the 
Lloyd Discounting Method is not recommended with this model. 

Again, no fixed rules exist for chosing the failure discounting parameters of the 
Straight Percent Discounting Method. Experience with the model, engineering 
judgement and possibly computer simulation will be necessary to select the proper 
combination of the discount interval and discount fraction. Even though the model is 
fairly robust concerning this selection, it is possible to choose parameters that will 
force the model to overestimate reliability. The danger of this occurring is greatest 
when expected growth in the system reliability does not occur. 

The reliability estimates produced by the exponential regression model exhibit a 
higher variance than those from the MLE With Discounting Model. This increase in 
variance is most pronounced in the early testing phases. Figures 4.13 and 4.14 are box 
plots of the one-hundred reliability estimates at each phase. They demonstrate the 
variability of the model. 
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Figure 4.7 MLE With Discounting 
Lloyd Discounting Method. 

As with any regression, several points are required to stabilize the regression 
parameter estimates. This causes highly variable responses in the first two or three 
phases of testing. This variance is largest when reliability in the early phases is low. In 
later phases and at higher reliability, the variance of estimates from- the Exponential 
Regression Model compares favorably to those of the MLE With Discounting Model. 

The Exponential Regression Mbdel closely tracks most reliability growth 
patterns. It is best when more than three test phases are planned. The most accurate 
results were obtained when the model was used with the Straight Percent Discounting 
Method. The modei is relatively robust with respect to the selection of discounting 
parameters; but, may still over or under estimate actual reliability if a bad combination 
of parameters is selected. The variance of reliability estimates in the early testing 
phases is large and makes the model undesirable for use with less than four testing 
phases. 
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Figure 4.8 MLE With Discounting 
Straight Percent Discounting Method. 
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Figure 4.9 MLE With Discounting 
Estimate Variability at Low Constant Reliability. 
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Figure 4.10 MLE With Discounting 
Estimate Variability With Increasing Reliability. 
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Figure 4.11 Exponential Regression Model 
Lloyd Discounting Method. 
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Figure 4.12 Exponential Regression Model 
Straight Percent Discounting Method. 
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Figure 4.13 Exponential Regression Model 
Estimate Variability at Low Constant Reliability. 
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EXPONENTIAL REGRESSION EST. 
STD DISCOUNTING N = 20. R = .75 
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Figure 4.14 Exponential Regression Model 
Estimate Variability With Increasing Reliability. 
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V. SUMMARY AND CONCLUSIONS 



A. SUMMARY 

The purpose of this study is to develop and evaluate discrete reliability growth 
models that employ fractional failure reduction, referred to as failure discounting. 
These models use all available data to estimate current reliability even if the actual 
reliability underlying all test data is not constant. This reduces the resources required 
to verify system reliability. This study focuses on developmental testing scenarios where 
testing is terminated after a fixed number of failures have been observed and the results 
of each trial are recorded as a success or failure, i.e. attribute data. 

Failure discounting is performed on past data to make it compatible with test 
data taken from the current system configuration. Most reliability estimators require a 
constant actual reliability for each trial to render an accurate estimate. Fractionally 
discounting past failures adjusts test data from previous testing on less reliable system 
configurations making the data useful in the computation of the current reliability 
estimate. 

Two different failure discounting methods were used in the study. The first, 
referred to as the Straight Percent Discounting Method allows the user to predetermine 
the fraction by which each failure will be reduced, the discount fraction, and an interval 
between applications of the discount Traction, the discount interval. The discount 
interval is the number of trials without a reoccurrence of the particular failure cause 
between application of the discount fraction. The second failure discounting method, 
referred to as the Lloyd Discounting Method, sets all past failures equal to the upper 
confidence bound for the probability of reoccurrence for the failure cause. Both failure 
discounting methods were used with each of the three discrete reliability growth models 
developed in the study. 

Three discrete reliability growth models using failure discounting were developed. 
The first model adds failure discounting to the maximum likelihood estimate for a 
proportion. The model is referred to as the Maximum Likelihood Estimate With 
Discounting Model. To use the model, the failure discounting method of choice is 
applied to the test data. The resulting discounted failures are returned to unity by 
adjusting the number of successful trials preceding each failure. The estimate of system 
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reliability is computed as the ratio of adjusted successful trials to adjusted total trials. 
Failure discounting allows the model to track changing reliability. 

The remaining two models are modifications of an exponential single phase 
reliability estimate of the form reliability = 1 — e^. One model uses regression to 
estimate A. This model is referred to as the Exponential Regression Model. The second 
model uses an unbiased estimate of A to produce a single testing phase reliability 
estimate. The current reliability estimate is a weighted average of the previous testing 
phase reliability estimate and the current single phase reliability estimate. The fraction 
of the total test trials to date used in the computation of each estimate becomes the 
weighting factor for the arithematic average. As with the Maximum Liklihood With 
Discounting Model, a failure discounting method is applied to the test data before 
either model is used to compute a reliability estimate. Both these models have the 
ability to track changing reliability without the use of failure discounting; but, failure 
discounting is employed to enhance the models performance. 

A Monte-Carlo simulation was written to evaluate the performance of the 
reliability growth models and failure discounting methods developed in the study. This 
was necessary because closed form solutions for the statistical properties of the models 
are not mathematically tractable. The simulation uses input parameters to generate a 
reliability growth pattern. Test data including the number of trials up to and including 
system failure and the cause of system failure are generated from the established 
reliability growth pattern. Each of the three reliability growth are applied to the 
simulated test data resulting in an estimate of system reliability at each testing phase. 
After this procedure is replicated a suitable number of times, estimates for the mean 
and standard deviation of each estimate are computed. These estimates are used to 
evaluate the performance of the reliability growth models. By varying the reliability 
growth pattern and the failure discounting parameters, the models performance over a 
wide range of situations may be evaluated. The conclusions reached from this analysis 
are included in the following section. 

B. CONCLUSIONS 

Even though the analysis of the reliability growth models performed in the study 
is not comprehensive enough to state conclusions with certainty, several general 
conclusions are apparent. 

The Lloyd Failure Discounting Method should not be used with the three 
reliability growth models developed in the study. With few exceptions, this method 
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caused the growth models to overestimate system reliability. This tendency to 
overestimate was particularly evident at low reliabilities when little or no actual 
reliability growth occurred. The few situations where the method performed well, rapid 
reliability growth, were handled equally well by the Straight Percent Discounting 
Method. The Straight Percent Discounting Method's flexibility makes the preferred of 
the two methods for fractionally discounting failures. 

The Weighted Average Reliability Growth Model, in its current form, should not 
be used to estimate system reliability. The model grossly overestimated reliability for 
most growth patterns tested. Even when no failure discounting was applied, the model 
overestimated system reliability. Adding either failure discounting method only 
increased the tendency to overestimate. 

Both the Maximum Liklihood Estimate With Discounting and the Exponential 
Regression Model exhibited performance superior to the standard, single phase MLE. 
estimate. But there is no clear cut choice between the Maximum Likelihood Estimate 
With Failure Discounting Model and the Exponential Regression Model. Both models 
accurately track a - wide range of reliability growth patterns. The MLE With 
Discounting Model appears to be the less variable of the two methods but is much 
more sensitive to the choice of failure discounting parameters than is the Exponential 
Regression Model. The variability of both models decreased with increasing reliability 
and additional test data. -For this reason the Exponential Regression Model is 
recommended for situations where four or more testing phases are planned and the 
system possesses high reliability with near certainty. For other cases the Maximum 
Likelihood Estimate With Failure Discounting is recommended because of its reduced 
variability. 

The discrete reliability grownh models using failure discounting developed in the 
study represent an improvement over conventional reliability estimation techniques for 
the special case of testing until a fixed number of failures have been observed with 
attribute data. However, these models can be improved. The following section 
contains recomendations for improvements and further study. 

C. RECOMENDATIONS 

The following are recomendations for improvement to the models introduced in 
this study. 

• Many more simulation runs with varied reliability growth patterns and failure 
discounting parameters are required before solid conclusions concerning the 
properties of the reliability growth models can be reached. 
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The Monte-Carlo Simulation should be modified so declining reliability can be 
modeled. 

An alternate weighting technique should be developed for the Weighted 
Average Model to correct its tendency to overestimate reliability. In certain 
situations, this model demonstrated the smallest variability of the three models 
developed. The model might prove to be an excellent reliability estimator if the 
overestimation tendency can be corrected. 

An additional failure discounting method that combines the flexibility of the 
Straight Percent Discounting Method with the non-arbitrary selection of the 
discounting fraction from the Lloyd method might prove useful. This could be 
accomplished by adding a discount interval to the current Lloyd Discounting 
Method. 

The possibility of using a simulation similar to the one developed here to aid in 
the selection of failure discounting parameters should be explored. Using an 
estimate of initial reliability, a reliability goal and the number of planned trials; 
the various choices for failure discounting parameters could be tested and one 
set chosen based on the simulation results. 

Further analysis of the performance of the reliability growth models with 
varying failure discounting parameters is required to develop "rules of thumb" 
for failure parameter selection. 



APPENDIX A 

DISCRETE RELIABILITY GROWTH MODEL SIMULATION 



I. FORTRAN CODE FOR THE SIMULATION 



k k k k k k k k k k k k k k k k k k k k k k k k k k k k kk k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k k 



k k 

* DISCRETE RELIABILITY GROWTH SIMULATION * 

k k 

* PROGRAMMED BY JAMES E DRAKE * 

* * 

* LAST MODIFIED 19 AUG 1987 * 

* X 



* 

* 

* 

k 

k 

k 

k 

* 

k 

* 

k 

k 

k 

k 

k 

k 

k 

k 

k 



THE FOLLOWING EXTERNAL FILES ARE USED BY THE PROGRAM 

INPUT : DATA AND PARAMETER INPUT FILE (DEVICE # 10) 

THESIS : OUTPUT FILE CONTAINING INTERMEDIATE COMPUTATIONS 
(DEVICE # 20) 

RELIAB : OUTPUT FILE CONTAINING FINAL RESULTS OF THE SIMULATION 
(DEVICE # 30) 

EST : OUTPUT FILE CONTAINING EACH PHASE ESTIMATE FOR EACH 
REPLICATION OF THE WEIGHTED AVERAGE ESTIMATE 
(DEVICE # 40) 

MLEWD : OUTPUT FILE CONTAINING MLE ESTIMATES USING DISCOUNTING 
FOR EACH PHASE AND EACH REPLICATION 
(DEVICE # 50) 

MLESP : OUTPUT FILE CONTAINING MLE ESTIMATE FOR EACH SINGLE PHASE 
AND ALL REPLICATIONS USING NO DISCOUNTING 
(DEVICE # 60) 

REGEST : OUTPUT FILE CONTAINING EACH PHASE ESTIMATE FOR EACH 
REPLICATION OF THE EXPONENTIAL REGRESSION ESTIMATE 
(DEVICE # 70) 



k 

k 

k 

k 

k 

k 

k 

k 



k 

k 

k 

k 

k 

k 

k 

k 

k 

k 
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* THE FOLLOWING IS A LIST OF KEY ARRAYS USED IN THE SIMULATION * 

* k 



k 

k 

k 

k 

k 

k 

k 

k 

k 

k 

k 

k 

k 

k 

k 

k 

k 

k 

k 



A 



NFAPH 

NFCAUS 

NTRIAL 

PHREST 



k 

k 

k 

k 

k 

k 

k 

k 

k 

k 



AREL 

YJK 

CUMSF 



MAIN WORKING ARRAY CONTAINS PROBABILITY OF SUCCESS FOR 
EACH FAILURE CAUSE , NUMBER OF TRIALS UNTIL FAILURE FOR 
EACH FAILURE CAUSE AND THE SYSTEM , CAUSE OF FAILURE , 
PHASE NUMBER , ADJUSTED NUMBER OF TRIALS AND ADJUSTED 
NUMBER OF FAILURES 

DIMENSION ( ((2*#CAUSES) +7) ^FAILURES ) 

CONTAINS THE NUMBER OF FAILURES IN EACH PHASE 
DIMENSION ( 1 , trPHASES ) 

BINARY ARRAY USED TO DETERMINE IF A FAILURE OCCURRED IN 
A PHASE 

DIMENSION ( 1 , ttFAILURE CAUSES) 

CONTAINS THE NUMBER OF TRIALS SINCE LAST FAILURE OR 
DISCOUNTING FOR EACH FAILURE CAUSE 
DIMENSION ( 1 , #FAILURE CAUSES ) 

RECORDS THE PHASE ESTIMATE FOR EACH ESTIMATOR WITHIN A 
SINGLE REPLICATION 
DIMENSION (4 , "PHASES ) 

ROW 1 : WEIGHTED AVERAGE ESTIMATE 

ROW 2 : MLE WITH DISCOUNTING 

ROW 3 : SINGLE PHASE MLE 

ROW 4 : EXPONENTIAL REGRESSION ESTIMATE 

CONTAINS ACTUAL SYSTEM RELIABILITY IN EACH PHASE 

DIMENSION (1 , #PHASES ) 

CONTAINS YJK VALUES UP TO 1000 
DIMENSION (1,1000) 

CONTAINS THE NUMBER OF SUCCESS AND FAILURES FOR EACH 
FAILURE CAUSE (USED WITH WEIGHTED AVERAGE EST.) 
DIMENSION (3 ,#FAILURE CAUSES) 

ROW 1 : NUMBER OF FAILURES 
ROW 2 : NUMBER OF SUCCESSES 



* 

k 

k 

k 
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k 
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* REG 

k 

* 
k 
* 

* 

k 

k 



ROW 3 : ADJUSTED NUMBER OF SUCCESSES * 
ARRAY USED TO COMPUTE THE EXPONENTIAL REGRESSION ESTIMATE * 
DIMENSION (5 ,#PHASES) * 
ROW 1 : K BAR * 
ROW 2 : Y BAR * 
ROW 3 : Y BAR FOR THE PHASE * 
ROW 4 : B HAT * 
ROW 5 : A HAT * 

* 



* THE REMAINING ARRAYS ARE USED TO COMPUTE THE MEAN AND VARIANCE 

* OF EACH ESTIMATE- AT EACH PHASE. THEY ALL HAVE THE SAME DIMENSIONS 

* AND STRUCTURE 

* DIMENSION (4 ,#PHASES) 

* ROW 1 : RUNNING SUM OF ESTIMATES 

* ROW 2 : RUNNING SUM OF SQUARED ESTIMATES 

* ROW 3 : MEAN OF THE ESTIMATES 

* ROW 4 : STANDARD DEVIATION OF THE ESTIMATES 

k 



k 

k 

k 

k 

k 

k 

k 

k 

k 



k EST 

* MLEWD 

* MLESP 

* REGEST 

k 



VALUES FOR THE WEIGHTED AVERAGE ESTIMATE 

VALUES FOR THE MLE WITH DISCOUNTING 

VALUES FOR THE SINGLE PHASE MLE 

VALUES FOR THE EXPONENTIAL REGRESSION ESTIMATE 



* 

* 

* 

* 

* 






C DEFINE AND DIMENSION VARIABLES 

PARAMETER (NR=50 ,NC=200) 

INTEGER REP , CUMSF , DISOPT 
REAL*4 MIN 

REAL*3 DSEED, MLESP. MLEWD, EST 

DIMENSION NFAPH(NR) , A(NR,NC) ,NFCAUS (NR) , NTRIAL(NR) , PHREST (4 ,NR) ,ES 
CT ( 4 , NR ) ,MLEWD(4,NR) ,MLESP(4,NR) ,REGEST(4,NR) , AREL(NR) , YJK(IOOO) , CU 
CMSF (3 ,NR) , REG( 5 , NR) 

C READ IN THE NUMBER OF CAUSES TO BE USED ( NCAUSE ) AND THE NUMBER 
C OF PHASES ( NPHASE ) IN THE TEST 

READ (10,*) NCAUSE 
RE AD (10,*) NPHASE 

C CREATE VARIABLES FOR THE ROW INDICES OF THE WORKING MATRIX ( A ) 

C IPHASE : PHASE 

C ISYSPR : ACTUAL COMPONENT RELIABILITY 

C INTR : NUMBER OF TRIALS UP TO AND INCLUDING FAILURE 
C IFAILC : CAUSE OF THE FAILURE 

C IADJFs ADJUSTED NUMBER OF FAILURES ED 

C AFTER DISCOUNTING HAS BEEN APPLIED 

C IADJT : ADJUSTED NUMBER OF TRIALS AFTER DISCOUNTING HAS BEEN APPLIED 
C IYJK : YJK COMPUTED ON THE ADJUSTED NUMBER OF TRIALS 

IPHASE = (2*NCAUSE)+1 
ISYSPR = IPHASE +1 
INTR = ISYSPR + 1 
IFAILC = INTR + 1 
IADJF = IFAILC +1 
IADJT = IADJF + 1 
IYJK = IADJT + 1 

C READ IN THE NUMBER OF FAILURES IN EACH PHASE ( NFAPH(I) ) AND 
C COMPUTE THE TOTAL NUMBER OF FAILURES IN THE TEST ( NFAIL ) 

NFAIL = 0 

DO 10 1=1, NPHASE 

READ (10,*) NFAPH(I) 

NFAIL = NFAIL + NFAPH(I) 

10 CONTINUE 

C INPUT THE PROBABILITY OF SUCCESS IN A SINGLE TRIAL FOR EACH CAUSE 
C IN THE FIRST PHASE 
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DO 20 1=1 ,NCAUSE 

READ (10 , *) A(I , 1 ) 

20 CONTINUE 

C INPUT THE REMAINING VARIABLES , THE NUMBER OF SUCCESSFUL TRIALS 
C BEFORE A DISCOUNT IS APPLIED (N) ; THE DISCOUNT FACTOR (R); THE SEED 
C FOR THE RANDOM NUMBER GENERATOR, GGUBFS , (DSEED) ; RELIABILITY 
C GROWTH FRACTION (FRIMP); TRIGGER FOR PRINTING INTERMEDIATE OUTPUT 
C ( IOPT ) 

C TRIGGERS FOR SAVING EACH ESTIMATE AT EACH PHASE FOR EACH ESTIMATOR 

C IOPT1 : WEIGHTED AVERAGE MODEL 

C IOPT2 : MLE WITH DISCOUNTING 

C IOPT3 : SINGLE PHASE MLE 

C I0PT4 : EXPONENTIAL REGRESSION MODEL 

C DISCOUNTING OPTION TRIGGER (DISOPT); LLOYDS FAILURE DISCOUNTING 
C PARAMETER (GAMMA) 

READ ( 10 , *) N 

read(io,*) R 

READ (10,*) DSEED 
READ ( 10,*) FRIMP 
READ ( 10 , *) NREP 
READ (10,*) IOPT 
READ (10,*) IOPT1 
READ i 10 ,* ) IOPT2 
READ (10,*) IOPT3 
READ (10,* I0PT4 
READ-: 10,*) DISOPT 
READ (10,*) GAMA 

XNREP = NREP 
DSEED1 = DSEED 

C INITIALIZE THE ARRAYS USED TO COMPUTE THE MEAN AND STANDARD DEVIATION 
C OF EACH ESTIMATOR 

DO 30 J=1 ,NPHASE 
DO 30 1=1,4 

EST ( I , J) = 0.0 
MLEWD(I,J) = 0.0 
MLESP(I , J) = 0.0 
REGEST(I , J) =. 0.0 
PHREST (I , J ) =0.0 

30 CONTINUE 

C COMPUTE AND STORE THE YJK VALUES UP TO 1000 

YJK(l) =0.0 
DO 40 1=1,999 

YJK( 1+1 ) = YJK(I) + 1.0/1 
40 CONTINUE 

C COMPUTE AND STORE K BAR FOR THE EXPONENTIAL REGRESSION MODEL 

SUM = 0.0 
DO 50 1=1 ,NPHASE 
SUM = SUM + I 
REG(1 ,1) = SUM/I 
50 CONTINUE 

C MAJOR REPETITION OF THE SIMULATION LOOP 
DO 500 REP=1 ,NREP 

C INITIALIZE FAILURE CAUSE VECTOR (NFCAUS) AND (CUMSF) 

C COMPUTE THE INITIAL SYSTEM RELIABILITY 

REL = 1. 

DO 60 1=1 ,NCAUSE 

NFCAUS ( I ) = 0 
REL = REL * A( I , 1 ) 

DO 60 J=1 .3 

CUMSF(J,I) = 0 

60 CONTINUE 
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c 



c 

c 



c 

c 



c 



c 

c 



c 

c 



c 



c 

c 

c 

c 



c 

c 



INITIALIZE COLUMN (FAILURE # ) COUNTER FOR THE WORKING ARRAY (A) 
J = 1 

LOOP TO COMPUTE THE NUMBER OF TRIALS UP TO AND INCLUDING FAILURE 
AND THE CAUSE OF FAILURE FOR EACH FAILURE IN EACH PHASE 

DO 130 K=1 ,NPHASE 

SKIP ACTUAL COMPONENT RELIABILITY COMPUTATION AFTER FIRST REP 
AND FOR FIRST FAILURE 

IF(J.EQ.l) GOTO 75 
IF (REP .GT . 1 ) GOTO 75 
REL = 1. 

COMPUTE NEW ACTUAL RELIABILITY FOR THE COMPONENT IN PHASE K 



DO 70 1=1 ,NCAUSE 

INCREASE CAUSE PR(SUCCESS) IF IT CAUSED FAILURE IN THE PREVIOUS PHASE 
COMPUTE NEXT PHASE RELIABILITY AND REINITIALIZE NFCAUS 



70 

75 



IF(NFCAUS(I) .EO.l 
A(I , J) = A(I , ( J-l 
ELSEIF (NFCAUS (I) .NE . 1 ) THEN 
A(I,J) = A(I, J-l)) 



THEN 

) +.((!• - A(I,(J-1)) )*FRIMP) 



ELSE 
END IF 

REL = REL*A(I , J) 
NFCAUS (I) = 0 
CONTINUE 
J1 = 1 
TRTOT =0.0 



COMPUTE THE NUMBER OF TRIALS UP TO AND INCLUDING FAILURE AND THE 
CAUSE OF FAILURE FOR EACH FAILURE IN THE PHASE 





DO 120 L=1 ,NFAPH(K) 
IF(REP.GT.l) GOTO 90 
IF(Jl.EO.l) GOTO 85 
DO 80 1=1 ,NCAUSE 

A(I,J) = A(I,(J-1)) 


80 


CONTINUE 


85 


A(ISYSPR, J) = REL 
A( IPHASE , J ) = K 


90 


MIN = 7.2E75 

DO 110 1=1 ,NCAUSE 


ASSIGN 


# TRIALS FOR CAUSES WITH PR (SUCCESS) 

IF(A(I,J).GE.l.) THEN 

A( (I+NCAUSE) ,J) = 7.2E75 
GOTO 100 

ELSEIF (A( I , J) .EQ.O. ) THEN 
A( (I+NCAUSE) ,J) = 1. 

GOTO 100 
ELSE 
ENDIF 



CONVERT UNIFORM (0,1) RANDOM VARIABLE TO GEOMETRIC (# TRIALS UNTIL 
FAILURE ) FOR EACH FAILURE CAUSE. RECORD THE MIN # TRIALS FOR THE 
CAUSES AS THE SYSTEM # TRIALS UP TO AND INCLUDING FAILURE AND 
RECORD THE FAILURE CAUSE 



100 



110 



A( (I+NCAUSE) , J) = INT ( 1 .+(LOG(GGUBFS (DSEED) ) /LOG(A( I , J) ) ) ) 
IF(A( ( I+MCAUSE ) , J) .LE .MIN) THEN 
MIN = A( (I+NCAUSE) , J) 

IMIN = I 
ELSE 
END IF 
CONTINUE 

A(IFAILC, J) = IMIN 
NFCAUS (IMIN) = 1 



COMPUTE THE TOTAL # OF TRIALS FOR THE MLE SINGLE PHASE ESTIMATE AND 
INCREMENT FAILURE # COUNTERS 
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A(INTR,J) = MIN 

TRTOT = TRTOT + A(INTR,J) 

J = J + 1 
J1 = J1 + 1 
120 CONTINUE 

C COMPUTE THE MLE ESTIMATE OF COMPONENT RELIABILITY FOR THIS PHASE AND 
C COMPUTE THE RUNNING SUM OF ESTIMATES AND THE SUM OF ESTIMATES SQUARED 
C FOR COMPUTATION OF THE MEAN AND STANDARD DEVIATION OF THE ESTIMATE 

PHREST(3 ,K) = (TRTOT - NFAPH(K) ) /TRTOT 
MLESP(1,K) = MLESP(1 ,K) + PHREST(3,K) 

MLESP(2,K) = MLESP (2 , K) + (PHREST(3 ,K)**2) 

130 CONTINUE 

C INITIALIZE THE ADJUSTED NUMBER OF FAILURES TO 1 AND THE COUNT OF THE 
C NUMBER OF TRIALS SINCE FAILURE OR DISCOUNTING (NTRIALS (I ) ) TO 0 
C IN PREPARATION FOR THE DISCOUNTING ROUTINE 

DO 140 J=1 ,NFAIL 

A(IADJF , J ) = 1. 

140 CONTINUE 

DO 150 1=1 ,NCAUSE 

NTRIAL(I) = 0 
150 CONTINUE 

C DISCOUNTING ROUTINE REVIEWS ALL PAST FAILURES AND CAUSES TO DATE 
C AND DETERMINES IF THE DISCOUNTING CONDITIONS HAVE BEEN MET. COMPUTES 
C THE ADJUSTED FAILURES, THE ADJUSTED # OF TRIALS AND YJK 

J = 0 

DO 300 K=1 ,N?HASE 
DO 200 L=1 ,NFAPH(K) 

J = J + 1 

C UPDATES THE NUMBER OF TRIALS SINCE FAILURE OR DISCOUNTING FOR EACH 
C FAILURE CAUSE 

ICAUSE = INT (A(IFAILC,J)+.5) 

DO 160 1=1 ,NCAUSE 

IF ( ICAUSE. EQ. I) THEN 
NTRIAL(I) = 0 
ELSEIF (ICAUSE .NE . I ) THEN 

NTRIAL(I) = NTRIAL ( I ) + INT ( A ( INTR , J ) + . 5 ) 

ELSE 
END IF 

160 CONTINUE 
200 CONTINUE 

C CHOOSE DISCOUNTING METHOD TO BE USED 
IF (DISOPT .NE .2) GOTO 130 
C PERFORM LLOYDS FAILURE DISCOUNTING METHOD 
DO 170 1=1 ,J 

II = INT(A(IFAILC , I )+ . 5 ) 

IF (NTRIAL ( I 1 ) . EQ . 0 ) THEN 
A(IADJF , I ) = 1.0 
GOTO 170 

ELSE 
END IF 



A(IADJF, I) = 1.0 - ( (1 . -GAMA)**(1 . 0/NTRIAL( II ) ) ) 
170 CONTINUE 
GOTO 210 

C PERFORMS STRAIGHT PERCENT FAILURE DISCOUNTING AND 
C COMPUTES THE ADJUSTED # OF FAILURES 

130 DO 190' 1=1, J 

II = INT (A(IFAILC,I)+.5) 

IF (NTRIAL (II ) .EQ.O) THEN 
A(IADJF , I ) = 1. 

ELSEIF (NTRIAL ( II ) . GE .N) THEN 
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A( IADJF , I ) = A(IADJF,I)*((1.-R)**(NTRIAL(I1)/N)) 

ELSE 

ENDIF 

190 CONTINUE 

C ADJUSTS THE # TRIALS SINCE FAILURE OR DISCOUNTING FOR THOSE CAUSES 
C THAT HAVE MET OR SURPASSED THE DISCOUNTING THRESHOLD 
C FOR THE STRAIGHT PERCENT DISCOUNTING METHOD 

DO 205 1=1 ,NCAUSE 

IF(NTRIAL(I).GE.N) NTRIAL ( I ) = MOD (NTRIAL ( I ) , N) 

205 CONTINUE 

210 TADJT =0.0 
TYJK =0.0 
TPYJK =0.0 
K1 = 0 

DO 215 12=1,3 
DO 215 1=1 ,NCAUSE 
CUMSF (12,1) = 0 
215 CONTINUE 

C COMPUTES THE ADJUSTED # OF TRIALS FROM THE ADJUSTED # OF FAILURES 
C AND COMPUTES THE SUM OF THE ADJUSTED # OF TRIALS FOR ESTIMATE COMP. 

PREL =0.0 
LTRIAL = 0 

DO 240 1=1, J 



A(IADJT , I ) = A ( INTR , I ) / A ( I AD JF , I ) 

TADJT = TADJT + A( IADJT, I) 

C COMPUTE YJK FROM THE ADJUSTED # OF TRIALS AND STORE THE SUM FOR 
C ESTIMATE COMPUTATION, USE ARRAY FOR # TRIALS < 1000 AND APPROX. FOR 
C VALUES > 1000 

N1 = NINT (A( IADJT , I ) ) 

' IF(Nl.LE.lOOO) THEN 

• A(IYJK,I) = YJK(Nl) 

ELSEIF ( N1 . GT . 1000 ) THEN 

A(IYJK,I) = (a ( IADJT , I )** . 10995 ) *3 . 55445 

ELSE 

ENDIF 

C DETERMINE IF A PHASE BOUNDARY HAS BEEN REACHED TO BEGIN ESTIMATE 
C COMPUTATION 

IF(I.EQ.l) GOTO 225 

IF(A(IPHASE,I).NE.A(IPHASE, (1-1))) THEN 
C COMPUTE THE WEIGHTED AVERAGE ESTIMATE 

MAX = 0 

K1 = K1 + 1 

C DETERMINE THE FAILURE CAUSE WITH THE LARGEST # OF FAILURES 

DO 220 11=1 ,NCAUSE 

IF (CUMSF (1 , II ) .GT .MAX) THEN 
MAX = CUMSF (1,11) 

ICOL = II 

ELSE 

ENDIF 

220 CONTINUE 



C COMPUTE YJK VALUE FOR THE CURRENT PHASE ESTIMATE 

IF(CUMSF(1, ICOL) .LE. 1000) THEN 
AHATL = YJK (CUMSF (1, ICOL)) 

ELSEIF (CUMSF (1 , ICOL) .GT.1000) THEN 
X = CUMSF (1 , .ICOL) 

AHATL = (X**.10995)*3. 55445 
ELSE 
ENDIF 

IX = CUMSF ( 1 , ICOL) + CUMSF (3 , ICOL) 
IF(IX.LE.IOOO) THEN 
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AHATU = YJK(IX) 

ELSEIF(IX. GT.1000) THEN 
X = IX 

AHATU = (X**.10995)*3. 55445 
ELSE 
END IF 

C COMPUTE CURRENT PHASE RELIABILITY ESTIMATE 

AHAT = AHATU - AHATL 
CREL = 1.0 - EXP ( -AHAT) 

X = CUMSF ( 1 , ICOL) + CUMSF(3 , ICOL) 

C COMPUTE AND STORE THE WEIGHTED AVERAGE ESTIMATE 

PREL = ((LTRIAL*PREL)/X) + ( ( (X-LTRIAL)*CREL)/X) 

LTRIAL = CUMSF ( 1 , ICOL) + CUMSF(3 , ICOL) 

C COMPUTE THE PHASE AND GLOBAL AVERAGE FOR YJK USED IN THE WOODS 
C REGRESSION ESTIMATES ARE 

REG(2,K1) = TYJK/ (1-1 ) 

REG(3,K1) = TPYJK/NFAPH(K1 ) 

TPYJK =0.0 

END IF 

C COMPUTE THE NUMBER OF FAILURES AND SUCCESSES FOR EACH FAILURE CAUSE 
C USED IN THE WEIGHTED AVERAGE ESTIMATE 

225 I CAUSE = INT(A(IFAILC , I)+ . 5 ) 

DO 230 11=1 ,NCAUSE 

CUMSF (2,11) = CUMSF (2,11) + INT(A(INTR, I) + .5) 

CUMSF(3 , II ) = CUMSF (3 , II ) + N1 
230 CONTINUE 

CUMSF (1,1 CAUSE) = CUMSF (1 , I CAUSE) + 1 
CUMSF (2 , ICAUSE) = CUMSF (2 , ICAUSE) - 1 
CUMSF (3 , ICAUSE ) = CUMSF (3 , ICAUSE) - 1 
TPYJK = TPYJK + A(IYJK.I) 

TYJK = TYJK + A(IYJK,I) 

240 CONTINUE 

C REPEAT COMPUTATIONS FOR THE WEIGHTED AVERAGE ESTIMATE FOR THE 
C FINAL PHASE 

MAX = 0 

K1 = K1 + 1 

DO 245 11=1 ,NCAUSE 

IF ( CUMSF (1 ,11) .GT.MAX) THEN 
MAX = CUMSF (1,11) 

ICOL = II 
ELSE 
END IF 

245 CONTINUE 

IF(CUMSF(1, ICOL) .LE. 1000) THEN 
AHATL = YJK(CUMSF (1 , ICOL) ) 

ELSEIF( CUMSF (1, ICOL) .GT.1000) THEN 
X = CUMSF ( 1 .ICOL) 

AHATL = (X**.10995)*3. 55445 
ELSE 
END IF 

IX = CUMSF (1 , ICOL) + CUMSF (3 , ICOL) 

IF (IX . LE . 1000 ) THEN 

AHATU = YJK (IX) 

ELSEIF(IX. GT.1000) THEN 
X = IX 

AHATU = (X**.10995)*3. 55445 
ELSE 
ENDIF 

AHAT = AHATU - AHATL 
CREL = 1.0 - EXP (-AHAT) 

X = CUMSF (1 , ICOL) + CUMSF (3 , ICOL) 

PREL = ( (LTRIAL*PREL)/X) + ( ( (X-LTRIAL)*CREL)/X) 
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LTRIAL = CUMSF(1 , I COL) + CUMSF(3 , ICOL) 

REG (2 , Kl) = TYJK/ ( J) 

REG (3 ,K1 ) = TPYJK/NFAPH(K1) 

PHREST(1 ,K) = PREL 

C COMPUTE THE MLE ESTIMATE OF PHASE RELIABILITY USING DISCOUNTING 
PHREST(2 ,K) = (TADJT - J)/TADJT 

C COMPUTE THE EXPONENTIAL REGRESSION ESTIMATE BEGINNING WITH B HAT 

SUM = 0.0 
SUMS =0.0 

IF (K.EQ.l) GOTO 252 
DO 250 I = 1 ( K 

SUM = SUM + ( (I-REG( 1 , K) )*REG(3 , I ) ) 

SUMS = SUMS + ((I-REG(l ; K))**2) 

250 CONTINUE 

REG(4 ,K) = SUM/SUMS 

C COMPUTE A HAT 

REG (5 ,K) = REG(2,K) - (REG(4 ,K)*REG(1 ,K) ) 

C COMPUTE AND STORE THE EXPONENTIAL REGRESSION ESTIMATE 

PHREST(4 , K) = 1.0 - EXP(-(REG(5,K) + (REG(4,K)*K) ) ) 
IF(PHREST(4,K) .LT.0.0) PHREST(4,K)=0 .0 
GOTO 255 

252 PHREST (4 , K) = 1.0 - EXP ( -REG(3 , 1 ) ) 

IF(PHREST (4 , K) .LT.0.0) ?KREST(4,K)=0 .0 

C STORE THE RUNNING SUM OF THE ESTIMATES FOR THE CURRENT PHASE AND THE 
C RUNNING SUM OF THE ESTIMATES SQUARED FOR COMPUTATION OF THE MEAN AND 
C STANDARD DEVIATION OF EACH ESTIMATE FOR EACH RELIABILITY GROWTH 
C MODEL 

255 EST(1,K) = EST(1,K) + PHREST(1,K) 

EST(2 ,K) = EST(2 ,K) + (PHREST(1 ,K)**2) 

MLEWD (1 ,K) = MLEWD ( 1 , K ) + PHREST(2,K) 

MLEWD(2,K) = MLEWD (2 ,K) + (PHREST(2 ,K)**2) 

REGEST(1,K) = REGEST( 1 ,K) + PHREST ( 4, K) 

REGEST(2,K) = REGEST(2,K) + (PHREST(4,K)**2) 

C STORE THE ACTUAL PHASE RELIABILITY 
AREL(K) = A(ISYSPR , J) 

C PRINT INTERMEDIATE OUTPUT IF REQUESTED AND THE NUMBER OF REPETITIONS 
C IS NOT GREATER THAN 5 

IF(IOPT.NE.l) GOTO 300 
IF (REP .GT . 5 ) GOTO 300 

WRITE (20, 1000) REP , K 

1000 FORMAT (T16, 'REPETITION NUMBER: ',14,' PHASE NUMBER: ',14) 

WRITE ( 20 ,1010 ) A(ISYSPR, J) 

1010 FORMAT(22X, 'ACTUAL COMPONENT RELIABILITY: ',F7.5) 

WRITE (20 , 1020 ) PHREST(1,K) 

1020 FORMAT (20X, 'PREDICTED COMPONENT RELIABILITY: *,F7.5) 

WRITE (20 ,1022 ) PHREST(2,K) 

1022 FORMAT 20X, 'MLE ESTIMATE USING DISCOUNTING: ',F7.5) 

WRITE (20 ,1025) PHREST(3,K) 

1025 FORMAT (18X, 'MLE ESTIMATE OF PHASE RELIABILITY: ',F7.5) 

WRITE (20 , 1027 ) PHREST(4,K) 

1027 FORMAT (14X, 'REGRESSION ESTIMATE OF PHASE RELIABILITY: ',F7.5) 

WRITE (20 , 1030 ) 

1030 FORMAT ( ' ' , ' ' ) 

DO 260 1=1 ,NCAUSE 

WRITE(20, 1035)1, A(I,J) ,A( (I+NCAUSE) , J) 

1035 FORMAT (12X, 'CAUSE: ',13,' PR(SUCCESS) : ' , F7 . 6 , ' # TRIALS : ', 

CF10.0) 

260 CONTINUE 

WRITE (20 , 1036 ) 

1036 FORMAT ( ' ' , ' ' ) 

WRITE (20, 1040) 
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1040 FORMAT (4X, 'FAIL #',3X,'FAIL CAUSE' ,3k, 1 # TRIALS 1 ,3X, 'AD J # FAIL', 3 
CK, ' ADJ # TRIALS' ,7X, 'YJK' ) 

DO 270 1=1, J 

WRITE (20 ,1050)1 ,A (IFAILC, I ) , A(INTR, I ) ,A(IADJF,I) ,A(IADJT,I ) ,A(IYJK 
C,I) 

1050 FORMAT (4X, 13 , 8X, F3 . 0 ,7X, F8 . 0 , 4X, F8 . 6 ,4X, F12 . 0 , 3X, Fll .4) 

270 CONTINUE 

WRITE(20 , 1060) 

1060 FORMAT ( ' ',///) 

300 CONTINUE 

C PRINT EACH OF THE 3 ESTIMATES TO THEIR APPROPRIATE OUTPUT FILE 
C IF REQUESTED 

IF(IOPTl.NE.l) GOTO 401 

400 WRITE(40 , 2000) (PHREST (1,1), I=1,NPHASE) 

401 IF (I0PT2 .ME . 1 ) GOTO 402 

WRITE(50 , 2000) (PHREST (2 , I ) , I=1,NPHASE) 

402 IF ( I0PT3 .NE . 1 ) GOTO 403 

WRITE (60 , 2000 ) (PHREST (3 , I ) , I=1,NPHASE) 

403 IF ( I0PT4 .NE . 1 ) GOTO 500 

WRITE (70 , 2000) (PHREST (4, I) , I=1,NPHASE) 

2000 FORMAT ( ' ' , 30 (F7 . 6 : IX) ) 

500 CONTINUE 

C UPON COMPLETION OF ALL REPETITIONS, COMPUTE THE MEAN AND STANDARD 
C DEVIATION OF EACH ESTIMATE FOR EACH PHASE SKIPPING COMPUTATIONS IF 
C ONLY ONE REPETITION IS REQUIRED 

IF (NREP.LE.l) GOTO 601 

DO 600 1=1 ,NPHASE 
EST (3 , I ) = EST(1 , I )/XNREP 
MLEWD (3,1) = MLEWD (1,1) /XNREP 
MLESP(3,I) = MLESP( 1,1) /XNREP 
REGEST(3 , I ) = REGEST( 1,1) /XNREP 

EST(4 , I ) = SQRT( (EST (2 , I ) - (XNREP* (EST (3 , I)**2) ) ) / (XNREP- 1 ) ) 

MLEWD (4, I) = SQRT ( (MLEWD (2 , I ) - (XNREP* (MLEWD (3 , I ) * A 2) ) ) / ( XNREP- 1 ) ) 
MLESP (4 , I ) = SQRT ( (MLESP (2 , I ) - (XNREP* (MLESP (3 , I )**2) ) ) / (XNREP- 1 ) ) 
REGEST(4, 1 )=SQRT( (REGEST(2 , I )- (XNREP*(REGEST (3 ,I)**2)))/ (XNREP- 1 ) ) 

600 CONTINUE 

C PRINT THE FINAL OUTPUT TABLE TO A FILE 

601 WRITE(30 ,3000) 

3000 FORMAT( 'O' ,T47 'DISCRETE RELIABILITY GROWTH SIMULATION') 

WRITE (30 , 3010; 

3010 FORMAT( '-' ,T54 'MODEL PARAMETER SUMMARY') 

WRITE (30, 3020) NCAUSE 

3020 FORMAT( 'O' ,T47, 'NUMBER OF POSSIBLE FAILURE CAUSES ',14) 

WRITE (30 , 3030) 

3030 F0RMAT('0' ,T38, 'CAUSE NUMBER ', T64 ,' SINGLE TRIAL PR( SUCCESS ) FOR 
CPHASE 1 ' ) 

DO 3050 M=l, NCAUSE 
WRITE(30,3040) M,A(M,l) 

3040 FORMAT (' ' ,T43 , 12 ,T79 , F8 . 6) 

3050 CONTINUE 

WRITE(30 , 3060) FRIMP 

3060 FORMAT( 'O' ,T37, 'FRACTION CAUSE RELIABILITY IMPROVES AFTER FAILURE 
C' , F3 . 6) 

WRITE (30, 3080) NPHASE 

3080 FORMAT ( '-' ,T48 ' NUMBER OF PHASES IN THE SIMULATION ',12) 
WRITE(30,3090) 

3090 FORMAT ( 'O' ,T42, 'PHASE NUMBER' ,T59 , 'NUMBER OF FAILURES IN THE FIRST 
C PHASE ' ) 

DO 3110 M=l, NPHASE 
WRITE (30 ,3100) M ,NFAPH(M) 

3100 FORMAT (' ' ,T43, 12 ,T73 , 12) 

3110 CONTINUE 

WRITE(30 ,3120) NFAIL 

3120 FORMAT( 'O' ,T51, 'TOTAL NUMBER OF FAILURES ',14) 

IF(DISOPT,EQ.2) GO TO 3160 
WRITE ( 30 , 3130) 
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3130 FORMAT ,T38, 'DISCOUNTING PERFORMED USING THE CONSTANT FRACTION 
CMETHOD 1 ) 

WRITE (30 ,3140) R 

3140 FORMAT ('O', T44 . 1 FRACTION EACH FAILURE IS DISCOUNTED ' ,F8.6) 

WRITE (30,3150) N 

3150 FORMAT ? 1 ' ,T33 , 'NUMBER OF TRIALS AFTER A FAILURE BEFORE A DISCOUNT 

C IS APPLIED 1 ,14) 

GO TO 3190 
3160 WRITE (30 , 3170 ) 

3170 FORMAT ( 1 ,T44. 'DISCOUNTING PERFORMED USING THE LLOYD METHOD') 

WRITE (30, 3 ISO) GAMA 

3180 FORMAT? 'O' ,T39, 'PERCENT C.I. ( USED AS DISCOUNT FRACTION ) ',F8.6 

C) 

3190 WRITE (30 , 3200) DSEED1 

3200 FORMAT? '-' ,T46. 'RANDOM NUMBER SEED USED ',F15.2) 

WRITE (30, 3210) NREP 

3210 FORMAT ('O', T37 , ' NUMBER OF REPETITIONS OF THE SIMULATION PERFORMED 

C 1 , 17 ) 

WRITE (30, 3220) 

3220 FORMAT? ' 1' ,T61 'ESTIMATOR:') 

WRITE (30, 3230) 

3230 FORMAT? 'O' ,T48' SINGLE' PHASE MLE WITHOUT DISCOUNTING') 

WRITE (30 , 3240 ) 

3240 FORMAT?'-' ,T60 'MEAN' ,T83, 'ESTIMATE' ,T109, '95 %' ) 

WRITE (30, 3250) 

3250 FORMAT?' ',T12,' PHASE NUMBER' ,T29 , 1 ACTUAL RELIABILITY' ,T52 , 1 PREDIC 
CTED RELIABILITY' ,T78, 'STANDARD DEVIATION 1 ,T101 ,' CONFIDENCE INTERVA 
CL 1 ) 

C COMPUTE C.I. FOR SINGLE PHASE MLE 

DO 3270 M=1 ,NPHASE 

Cl = ( 1 . 96*MLESP (4 ,M) ) / SQRT (XNREP) 

CIU = MLESP(3,M) + Cl 
CIL = MLESP?3.M) - Cl 

WRITE (30 ,3260 ) M, AREL(M) ,MLESP (3 ,M) ,MLESP (4 ,M) , CIL , CIU 
3260 FORMAT’? 'O' ,T17 , 12 ,T34 , F8 .6 ,T58 , F8.6 ,T82 , F9 . 6 ,T99 , ' ( ' , F3 . 6 , ' , ',F 
C8 . 6 , ' )') 

3270 CONTINUE 

WRITE (30 ,3220) 

WRITE (30 ,3280) 

3280 FORMAT? 'O' ,T42, 'MAX LIKELIHOOD ESTIMATE USING DISCOUNTED FAILURES' 
C) 

WRITE (30, 3240) 

WRITE ?30 , 3250 ) 

C COMPUTE C.I. FOR MLE WITH DISCOUNTING 

DO 3290 M=1,NPHASE 

Cl = ( 1 . 96*MLEWD (4 ,M) ) / SQRT (XNREP ) 

CIU = MLEWD ( 3 , M ) + Cl 
CIL = MLEWD?3.M) - Cl 

WRITE (30 , 3260) M, AREL(M) ,MLEWD(3,M) ,MLEWD(4 ,M) , CIL , CIU 
3290 CONTINUE 

WRITE(30,3220) 

WRITE ? 30 , 3300 ) 

3300 FORMAT? 'O' ,T38, 'WEIGHTED AVERAGE ESTIMATE USING FAILURE DISCOUNTIN 
CG' ) 

WRITE (30 ,3240) 

WRITE ?30 ,3250) 

C COMPUTE C.I. FOR WEIGHTED AVERAGE ESTIMATES 

DO 3310 M=1 ,NPHASE 

Cl = ( 1 . 96*EST(4 ,M) ) /SORT (XNREP) 

CIU = EST(3,M) + Cl 
CIL = EST?3,M) - Cl 

WRITE(30,3260) M, AREL (M) , EST(3 ,M) , EST(4 ,M) , CIL , CIU 
3310 CONTINUE 

WRITE (30 , 3220) 

WRITE?30 ,3320) 

3320 FORMAT? 'O' ,T43, 'REGRESSION ESTIMATE USING DISCOUNTED FAILURES') 
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WRITE(30,3240) 

WRITE (30 , 3250) 

C COMPUTE C.I. FOR EXPONENTIAL REGRESSION ESTIMATES 
DO 3330 M=1,NPHASE 

Cl = (1 . 96*REGEST(4,M) ) /SQRT(XNREP) 

CIU = REGEST(3 ,M) + Cl 
CIL = REGEST(3,M) - Cl 

WRITE (30, 3260) M, AREL(M) ,REGEST(3 ,M) ,REGEST(4,M) ,CIL,CIU 
3330 CONTINUE 

WRITE (30 ,3340) 

3340 FORMAT ( 1 1 1 ,T59' RECAPITULATION '// ) 

WRITE( 30 , 3350) 

3350 FORMAT ( ' - ' ,T3, 'PHASE 1 ,T11, 'ACTUAL 1 ,T28, 'MEAN' ,T38, 'EST' ,T53, 'MEAN' 

C ,T63 , 'EST' , T78 , 'MEAN' ,T83, 'EST' ,T103, 'MEAN' ,T113, 'EST' ) 

WRITE (30 , 3360) 

3360 FORMAT ( ' ' ,T11, ' RELIAB ' ,T28, 1 WGT ' ,T38, 'STD' ,T53, ' MLE ' ,T63, 'STD' ,T7 
C7 , 'PHASE' ,TS8. 'STD' ,T103, 'REG' ,T113, 'STD' ) 

WRITE (30 ,3370) 

3370 FORMAT ( ' ' ,T28, 'AVG' ,T35, 'DEVIATION' ,T53, ' W/D ' ,T60, 'DEVIATION' ,T78 
C, 'MLE' , T85 , 'DEVIATION' ,T103, 'EST' ,T110, 'DEVIATION' ) 

WRITE (30 , 3375 ) 

3375 FORMAT (' ',T23,'EST'/) 

DO 650 1=1 ,NPHASE 

WRITE ( 30 , 3380 ) I , AREL(I ), EST (3, I), EST (4, I) ,MLEWD(3,I) ,MLEWD(4,I) , 
CMLES? (3 , I ) ,MLESP (4 , I ) ,REGEST(3,I) ,REGEST(4,I) 

3380 FORMAT ( ' 0 1 , T4 , 13 , Til , ?7 . 6 , T26 , F7 . 6 , T36 , F7 . 6 , T5 1 , F7 . 6 , T61 , F7 . 6 , T76 , 
CF7 . 6 , T86 , F7 . 6 ,T101 , F7 . 6 , Till , F7 . 6 ) 

650 CONTINUE 



STOP 

END 
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2. SAMPLE SIMULATION INPUT FILE 



5 NUMBER OF FAILURE CAUSES 

10 NUMBER OF PHASES ( NPHASE ) 

1 NUMBER OF FAILURES IN PHASE 1 

1 NUMBER OF FAILURES IN PHASE 2 

1 NUMBER OF FAILURES IN PHASE 3 

1 NUMBER OF FAILURES IN PHASE 4 

1 NUMBER OF FAILURES IN PHASE 5 

1 NUMBER OF FAILURES IN PHASE 6 

1 NUMBER OF FAILURES IN PHASE 7 

1 NUMBER OF FAILURES IN PHASE 8 

1 NUMBER OF FAILURES IN PHASE 9 

1 NUMBER OF FAILURES IN PHASE 10 

.8 1 - PROB. OF OCCURRENCE FOR CAUSE 1 IN PHASE 1 

.8 1 - PROB. OF OCCURRENCE FOR CAUSE 2 IN PHASE 1 

.8 1 - PROB. OF OCCURRENCE FOR CAUSE 3 IN PHASE 1 

.7 1 - PROB. OF OCCURRENCE FOR CAUSE 4 IN PHASE 1 

.7 1 - PROB. OF OCCURRENCE FOR CAUSE 5 IN PHASE 1 

50 DISCOUNT INTERVAL FOR STRAIGHT PERCENT DISCOUNTING 

.75 DISCOUNT FRACTION FOR STRAIGHT PERCENT DISCOUNTING 

624712.0 RANDOM NUMBER SEED 

.95 RELIABILITY GROWTH FRACTION 

10000 NUMBER OF DESIRED REPETITIONS FOR THE SIMULATION 

0 INTERMEDIATE OUTPUT OPTION (l.INT. OUTPUT; 0: NO OUTPUT) 

0 SAVE ALL WEIGHTED AVERAGE ESTIMATES (1: YES; 0: NO ) 

0 SAVE ALL MLE W/ DISCOUNTING ESTIMATES (1: YES; 0: NO ) 

0 SAVE ALL MLE SINGLE PHASE ESTIMATES (1: YES; 0: NO ) 

0 SAVE ALL REGRESSION ESTIMATES (1: YES; 0: NO ) 

2 DISCOUNTING OPTION (1: STRAIGHT % ; 2: LLOYD METHOD) 

.2 GAMMA FOR LLOYD DISCOUNTING METHOD 
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3. SAMPLE SIMULATION OUTPUT FILE 



DISCRETE RELIABILITY GROWTH' SIMULATION 



NUMBER 


HOOEL PARAMETER SUMMARY 
OF POSSIBLE FAILURE CAUSES 5 


CAUSE NUMBER 


SINGLE TRIAL PRC SUCCESS ) FOR PHASE I 


* 1 


0.300000 


2 


0.300000 


3 


0.300000 


A 


0.700000 


5 


• 0.700000 



FRACTION CAUSE RELIABILITY IMPROVES AFTER FAILURE 0.950000 



NUMBER OF PHASES IN THE SIMULATION 10 



PHASE NUMBER 

1 

2 
3 
A 

5 

6 
7 
3 
9 

IQ 



NUMBER OF FAILURES IN THE FIRST PHASE 
1 
I 
1 
1 
l 
I 
I 
1 
1 
I 



TOTAL NUMBER OF FAILURES IQ 



DISCOUNTING PERFORMED USING THE CONSTANT FRACTION METHOO 

FRACTION EACH FAILURE IS DISCOUNTED 0.750000 
NUMBER OF TRIALS AFTER A FAILURE BEFORE A DISCOUNT IS APPLIES 50 

RANDOM NUMBER SEED USED 42A7I2.00 

NUMBER OF REPETITIONS OF THE SIMULATION PERFORMED 100 
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APPENDIX B 

GRAPHICAL RESULTS OF THE RELIABILITY GROWTH MODEL SIMULATION 
1. MEAN ESTIMATES FOR EACH ESTIMATOR WITH CONSTANT PARAMETERS 



STD DISCOUNTING, N = 5 ; F = .25 



— ACTUAL RELIABILITY 
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85 



STD DISCOUNTING, N = 5 ; F = .50 

o 

— ACTUAL RELIABILITY 
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1 MLE WITH DISCOUNTING 
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STD DISCOUNTING, N = 10 ; F = .25 
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STD DISCOUNTING, N = 10 ; F = .50 

o 



— ACTUAL RELIABILITY 
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1 — MLE WITH DISCOUNTING 
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— EXPONENTIAL REGRESSION EST. 
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STD DISCOUNTING, N = 10 ; F = .75 
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LLOYD DISCOUNTING; GAMMA = .8 
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LLOYD DISCOUNTING; GAMMA = .999 
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LLOYD DISCOUNTING; GAMMA = .99999 
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STD DISCOUNTING, N = 20 ; F = .25 

o 




to 

o 



cn 

< 

Id 

(X 



o 



— • — actual RELIAOILITY 

WEIGHTED AVERAGE ESTIMATE 
— h — MLE WITH DISCOUNTING 
— > <— SINGLE PHASE MLE 
—■9— EXPONENTIAL REGRESSION EST. 



CM 

6 



o 



J L 

2 



J___L 

6 



X 

8 



PHASE 



-L 

10 



90 



CO 

o 



STD DISCOUNTING, N = 20 ; F = .50 
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a 


STD DISCOUNTING. N = 50 ; F = .50 
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LLOYD DISCOUNTING; GAMMA = .8 
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LLOYD DISCOUNTING; GAMMA = .999 
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3. ESTIMATOR VARIABILITY ( MULTIPLE BOX PLOTS ) 
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